elasticsearch


logstash retrying failed action with response code: 429


ES:
node1 -node10 . 10 machines
cpu: 32 core
jvm: 60G
logstash:
logstash-node1 , logstash-node2
cpu:32 core
jvm 20G
4 logstsh app in every machine
datasource: 18w/s-20w/s
dataformat :
"index" : {
"number_of_shards" : 8,
"number_of_replicas" : 0,
"refresh_interval" : "30s"
}
"filed1" : {
"type": "integer"
},
"filed2" : {
"type": "integer"
},
"filed3" : {
"type": "long"
},
"filed4" : {
"type": "text"
},
"filed5":{
"type": "text"
},
"filed6":
{
"type": "text"
},
"fild7":{
"type": "date"
},
"field8":{
"type": "geo_point"
}
}
logstash.yml
//This defaults to the number of the host's CPU cores.
pipeline.workers: 20
//How many workers should be used per output plugin instance
pipeline.output.workers: 20
//How many events to retrieve from inputs before sending to filters+workers
pipeline.batch.size: 9000
logstash kafka.conf
input{
kafka {
codec => "json"
topics => ["mytopic"]
bootstrap_servers => "kafka1:9092,kafka2:9092,kafka3:9092,kafka4:9092,kafka5:9092,kafka6:9092,kafka7:9092,kafka8:9092,kafka9:9092,
kafka11:9092,kafka12:9092,kafka13:9092,kafka14:9092,kafka15:9092,kafka16:9092,kafka17:9092,kafka18:9092,
kafka19:9092"
client_id => "kafka"
auto_offset_reset => "latest"
group_id => "wzfw510_3"
consumer_threads => 20
auto_commit_interval_ms => 500
}
}
output{
elasticsearch{
#codec => rubydebug
hosts => ["node1:8200","node2:8200","node3:8200","node4:8200","node5:8200","
node6:8200","node7:8200","node8:8200","node9:8200","node10:8200"]
index => "myindex"
document_type => "mytype"
flush_size => 8000
#idle_flush_time => 1
}
}
Q:
Now the speed is 800w-900w/min
the speed is slow ,
and logstash always output error :
[2017-05-11T09:27:57,327][INFO ][logstash.outputs.elasticsearch] retrying failed action with response code: 429 ({"type"=>"es_rejected_execution_exception", "reason"=>"rejected execution of org.elasticsearch.transport.TransportService$7#2393d3d9 on EsThreadPoolExecutor[bulk, queue capacity = 50, org.elasticsearch.common.util.concurrent.EsThreadPoolExecutor#63898a2e[Running, pool size = 32, active threads = 32, queued tasks = 50, completed tasks = 27163960]]"})
How to improve the logstash speed . thks.

Related Links

Default elasticsearch configuration for docker container
Add alias to index from template?
Save ElasticSearch snapshot physical files in specific location
Run elasticsearch with docker
elasticsearch: How to reinitialize a node?
Elasticsearch filter on mulitple nested paths
Need to Sort the _term in elastic search aggregation result
Why search_analyzer (keyword) does not work with edgeNgram index_analyer?
Elasticsearch startup: not available immediately
Elasticsearch Partial Fields With Inner Hits
ElasticSearch issue when adding new path.data
To copy an index from one machine to another in elasticsearch
Elasticsearch 2.0: how to delete by query in Java
Erroneour match using snowball analyzer
ElasticSearch 2.0 upgrade now can't connect to server
Elasticsearch layered ordering

Categories

HOME
orientdb
applescript
devexpress
compilation
adobe
gallery
log4j2
yeoman-generator-angular
heuristics
parsley.js
multi-step
live-streaming
sslhandshakeexception
django-rq
heap-dump
asciimath
android-custom-view
spyder
error-logging
hdl
rxjs5
software-packaging
ratio
raima
unordered-multimap
data-collection
plyr
bitbake
unmarshalling
monitor
progid
windows2012
counting
fastlane
unification
flowchart
robust
xcode8.2
daz3d
winston
android-webservice
webvtt
htmlspecialchars
pen
algebra
mongodb-aggregation
apache-modules
pbx
researchkit
bigdecimal
bilinear-interpolation
nvda
callouts
android-syncadapter
apache-pig-grunt
maximize
jemdoc
mnesia
gulp-uglify
taco
onepage-checkout
django-validation
webgrind
pack
phpquery
boost-proto
nastran
dsa
supercomputers
360-degrees
domino-designer-eclipse
android-audiomanager
createfile
fat32
rebar
adwords-apiv201402
berkeley-db-xml
two.js
forio-contour
cidr
windows-taskbar
typesafe-stack
lsa
postgres-xc
cyrillic
gprof
system-requirements
onconfigurationchanged
uipangesturerecognizer
zend-server-ce
botnet
sql-parametrized-query
aggregator
android-input-method
.net-services
facebook-fbml
fuzzy-comparison
gtktextview
chronic
jvm-bytecode

Resources

Encrypt Message