elasticsearch


ElasticSearch assign own IDs while indexing with LogStash


I am indexing a large corpora of information and I have a string-key that I know is unique. I would like to avoid using the search and rather access documents by this artificial identifier.
Since the Path directive is discontinued in ES 1.5, anyone know a workaround to this problem!?
My data look like:
{unique-string},val1, val2, val3...
{unique-string2},val4, val5, val6...
I am using logstash to index the files and would prefer to fetch the documents through a direct get, rather than through an exact-match.
In your elasticsearch output plugin, just specify the document_id setting with a reference to the field you want to use as id, i.e. the one named 1 in your csv filter.
input {
file {...}
}
filter {
csv{
columns=>["1","2","3"]
separator => ","
}
}
output {
elasticsearch {
action => "index"
host => "localhost"
port => "9200"
index => "index-name"
document_id => "%{1}" <--- add this line
workers => 2
cluster => "elasticsearch-cluster"
protocol => "http"
}
}

Related Links

elastic search autocomplete, searching with space creating an issue
(ELK) logstash grok for SBR accounting log
aws cloudsearch/lucene query street names
Getting cardinality of multiple fields?
Aggregating a Key/Value list in ElasticSearch
“reverse cardinality” in elasticsearch?
ElasticSearch- Using Fields doesn't return any documents on Nest
Analyzer to find , e.g: “starbucks” when mistakenly querying “star bucks”
Elasticsearch - boost document based on field's specific value
How to get elasticsearch most used words?
Umlaut in Elastic Suggesters
Index creation move elastic search cluster to red
Is there multiword synonyms with slop in ES
Elasticsearch: How do you delete a mapping type without deleting an entire index?
What are the performance drawbacks of flat documents vs. nested ones?
Template does not exists for cookbook elasticsearch

Categories

HOME
amazon-web-services
class
foaf
haproxy
hid
message-queue
risk-management
capistrano
swarm
getopenfilename
colors
smooks
google-sites
boomi
pymongo
channel
candlestick-chart
substring
data-collection
uilocalnotification
points
bonita
mdns
cmis
spring-rabbitmq
xvfb
identify
microsoft-ui-automation
mars-simulator
sspi
utf
angular-datatables
pyttsx
flume-twitter
django-static-precompiler
libreoffice-writer
ol3-google-maps
password-hash
ilrepack
sigsegv
punctuation
emf-compare
rational-performance-test
always-on-top
dpkg
jtag
odp.net-managed
gwt-syncproxy
taco
gdl
miniconda
opencyc
screen-orientation
vdm-sl
modern.ie
alwayson
relational-model
web-api
kotlin-android-extensions
fuseesb
ssmtp
jolie
configurable-product
global-scope
nesper
producer
lmax
accessory
wso2bam
spring-security-acl
smips
ggts
apportable
mongo-jackson-mapper
asp.net-mvc-3-areas
multiscaleimage
multiple-dispatch
story
clearinterval
popup-blocker
fitch-proofs
dropshadow
net-use
rtd
css-friendly
mongrel2
bass
multiple-users
iphone-sdk-4.3
self-tracking-entities
ppc
inversion
rootkit

Resources

Encrypt Message



code
soft
python
ios
c
html
jquery
cloud
mobile