elasticsearch


Plot a Tile map with the ELK stack


I'm trying to create a tile map with Kibana. My conf file logstash works correctly and generates all what Kibana needs to plot a tile map. This is my conf logstash :
input {
file {
path => "/home/ec2-user/part.csv"
start_position => "beginning"
sincedb_path => "/dev/null"
}
}
filter {
csv {
separator => ","
columns => ["kilo_bytes_total","ip","session_number","request_number_total","duration_minutes_total","referer_list","filter_match_count_avg","request_number_avg","duration_minutes_avg","kilo_bytes_avg","segment_duration_avg","req_by_minute_avg","segment_mix_rank_avg","offset_avg_avg","offset_std_avg","extrem_interval_count_avg","pf0_avg","pf1_avg","pf2_avg","pf3_avg","pf4_avg","code_0_avg","code_1_avg","code_2_avg","code_3_avg","code_4_avg","code_5_avg","volume_classification_filter_avg","code_classification_filter_avg","profiles_classification_filter_avg","strange_classification_filter_avg"]
}
geoip {
source => "ip"
database => "/home/ec2-user/logstash-5.2.0/GeoLite2-City.mmdb"
target => "geoip"
add_field => [ "[geoip][coordinates]", "%{[geoip][longitude]}" ]
add_field => [ "[geoip][coordinates]", "%{[geoip][latitude]}" ]
add_tag => "geoip"
}
mutate {
convert => [ "[geoip][coordinates]", "float"]
}
}
output {
elasticsearch {
index => "geotrafficip"
}
}
And this is what that generates :
It looks cool. Trying to create my tile map, I have this message :
What to do ?
It seems that I must add somewhere the possiblity to use dinamyc templates.. Should I create a template and add it to my file conf logstash ?
Can anybody give me some feedback ? thx !
If you look in the Kibana settings for your index, you'll need at least one field to show up with a type of geo_point to be able to get anything on a map.
If you don't already have a geo_point field, you'll need to re-index your data after setting up an appropriate mapping for the geoip.coordinates field. For example: http://stackoverflow.com/a/42004303/2785358
If you are using a relatively new version of Elasticsearch (2.3 or later), it's relatively easy to re-index your data. You need to create a new index with the correct mapping, use the re-index API to copy the data to the new index, delete the original index and then re-index back to the original name.
You are using the geoip filter wrong and are trying to convert the longitude and latitude to float. Get rid of your mutate filter and change the geoip filter to this.
geoip {
source => "ip"
fields => ["latitude","longitude"]
add_tag => "geoip"
}
This will create the appropriate fields. And the required GeoJSON object.

Related Links

Elasticsearch NEST client singleton usage
Elasticsearch: suggest users based on likes
Set every property type to not_analyzed for custom object
How to tell ElasticSearch to create nested fields
Elasticsearch minimum master nodes
How to create an Elasticsearch index without word-splitting?
Run a simple sql group by query in kibana 4
Clearing ttl from an object in elasticsearch
how can i search for values that have “-” dash in them with elastic search
Elasticsearch array of query strings
ElasticSearch Update Multi-field Mapping
Referencing to another index in elasticsearch
Elasticsearch : Need advice on architectural design of my cluster
What is pro/con of having big ES index or several small ES indexs on same data?
Elasticsearch get works half of the time
Scoring by term position in ElasticSearch?

Categories

HOME
jboss
spring-data
extjs
apache-nifi
macos-sierra
matrix
weblogic12c
xcode
windows-8.1
ontouchlistener
yahoo-weather-api
bots
activemq
hid
biztalk
semantic-web
autocomplete
gallery
ontology
haxe
javafx-8
cocotb
yeoman-generator-angular
amazon-emr
live-streaming
spring-jpa
dma
c#-7.0
vertica
varnish-vcl
pe
anova
coreldraw
sharedpreferences
altium-designer
conda
perfino
mod-wsgi
jpa-2.1
python-appium
strophe.js
sparkle
phpstorm-2016.1
publishing
lxml
ms-media-foundation
adminer
cmis
testlink
llvm-ir
rdp
python-sounddevice
uipickerview
newtons-method
xcode8.2
audio-converter
spongycastle
tinyioc
icecast
knockout-2.0
clrs
polymaps
emf-compare
boost-compute
mifos
ndk-build
openquery
textblock
jsr363
wepay
textpattern
bjam
gitlab-omnibus
roxygen2
mobilefirst-server
artisan
gflags
mutators
directorysearcher
fasterxml
n-tier-architecture
password-recovery
loginview
communicate
nachos
django-settings
vlab
jquery-transit
disjoint-union
html-frames
background-thread
gprof
user-forums
spark-view-engine
clrprofiler
hardcode
boost-foreach
startupscript
kohana-auth
channelfactory
mongrel
odac
aspbutton
efs
regioninfo
jvm-bytecode

Resources

Encrypt Message