elasticsearch


Error 429 [type=reduce_search_phase_exception]


I have many languages for my docs and am following this pattern: One index per language. In that they suggest to search across all indices with the
/blogs-*/post/_count
pattern. For my case I am getting a count across the indices of how many docs I have. I am running my code concurrently so making many requests at same time. If I search
/blogs-en/post/_count
or any other language then all is fine. However if I search
/blogs-*/post/_count
I soon encounter:
"Error 429 (Too Many Requests): [reduce] [type=reduce_search_phase_exception]
"
Is there a workaround for this? The same number of requests is made regardless of if I use
/blogs-en/post/_count or /blogs-*/post/_count.
I have always used the same number of workers in my code but re-arranging the indices to have one index per language suddenly broke my code.
EDIT: It is a brand new index without any documents when I start the program and when I get the error I have about 5,000 documents so not under any heavy load.
Edit: I am using the mapping found in the above-referenced link and running on a local machine with all the defaults of ES...in my case shards=5 and replicas=1. I am really just following the example from the link.
EDIT: The errors are seen with as few as 13-20 requests are made and I know ES can handle more than that. Searching /blogs-en/post/_count instead of /blogs-*/post/_count, etc.. can easily handle thousands with no errors.
Another Edit: I have removed all concurrency but still can only access 40-50 requests before I get the error.
I don't get an error for that request and it returns total documents.
Is you'r cluster under load?
Anyway, using simple aggregation you can get total document count in hits.total and per index document count in count_per_index part of result:
GET /blogs-*/post/_search
{
"size": 0,
"query": {
"match_all": {}
},
"aggs": {
"count_per_index": {
"terms": {
"field": "_index"
}
}
}
}

Related Links

Elasticsearch function_score query
Elasticsearch 2.0.0 cluster zen discovery in docker
How can I use Kafka to retain logs in logstash for longer period?
Optimal way to set up ELK stack on three servers
Cluster Level Logging with Elasticsearch and Kibana does not work in kubernetes
ElasticSearch Couchbase Replication Issue
How to query for inner_hits against grandparents in multi-generational setup
Request timedout during delete/create on elasticsearch while snapshot is being taken
how to use two parallel Aggregation for elasticsearch nest
How do I enable remote access/request in Elasticsearch 2.0?
elasticsearch faking index per user - how are routing values inferred when updating?
elasticsearch: Proper config in 3 node cluster for each node to have full copy of index?
Multiple Nested Aggregations in Elasticsearch
Reconnection in Elasticsearch Cluster
Elasticsearch - Has_Parent or Has_Child query returning empty results
Elasticsearch search body from URL search

Categories

HOME
class
typelite
mousemove
plaid
ios-simulator
typeerror
yql
visualforce
javafx-8
angular2-template
pyresttest
responsive-design
match
smooks
complexity-theory
mailgun
async-await
twisted
heap-dump
rest-assured
l20n
mixpanel
akka-persistence
bsd
kamailio
sybase-ase
dragula
plyr
spreadsheetgear
xdebug
openbravo
google-play-developer-api
dotcms
source-insight
tightvnc
vcenter
osx-lion
cargo
elasticsearch-aggregation
comm
redisson
sonatype
glimpse
case-when
kendo-treeview
portal
streamwriter
nclam
data-management
pyaudio
maven-release-plugin
noclassdeffounderror
codecharge
persistent-object-store
gridbaglayout
gwt-syncproxy
ioctl
decidable
calcite
knife
modern.ie
org-babel
actionpack
dbmigrate
rfc5545
nsmutabledata
baseadapter
mod-auth-openidc
dnsjava
storing-data
usb-flash-drive
django-facebook
form-for
hints
insertion
spiral
office-app
ckeditor.net
sfinae
hardcode
autosize
opends
microblogging
mmc3
dynamic-c
database-deadlocks
suphp
solandra
ccnet-config
android-input-method
iphone-sdk-3.1.3
community-server
service-factory

Resources

Encrypt Message