elasticsearch


Need to Sort the _term in elastic search aggregation result


Need to Sort the _term in elastic search aggregation result
As per the above question, my elastic search query with the script given by the Val worked fine till the last week. We have upgraded the ES version and suddenly it stopped working.
{
"size": 0,
"aggs": {
"count": {
"terms": {
"script": "doc.billingSequence.value as Integer", <--- transform the terms to integers
"order": {
"_term": "asc"
},
"value_type": "integer", <--- consider the terms as integer when sorting
"size": 10
}
}
}
}
Now suddenly my ES stopped working with code containing "as Integer" in script. Can anyone please check and help.
[DEBUG] 2015-10-26 10:00:13,907 org.elasticsearch.action.search.type - [Eson the Searcher] [owce_assets][4], node[aNnn7tvHRi6tCkOA-RU2nw], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest#123872f5] lastShard [true]
org.elasticsearch.search.SearchParseException: [owce_assets][4]: from[0],size[100]: Parse Failure [Failed to parse source [{"from":0,"size":100,"aggregations_binary":"ewoJCQkJCQkiY291bnQiOiB7IAoJCQkJCQkJInRlcm1zIjogewoJCQkJCQkJCSJzY3JpcHQiOiJkb2MuYmlsbGluZ1NlcXVlbmNlLnZhbHVlIGFzIEludGVnZXIiLAoJCQkJCQkJCSJvcmRlciI6IHsKCQkJCQkJCQkgICJfdGVybSI6ICJhc2MiCgkJCQkJCQkJfSwKCQkJCQkJCQkidmFsdWVfdHlwZSI6ICJpbnRlZ2VyIiwgICAgIAoJCQkJCQkic2l6ZSI6IDIwCgkJCQkJCQkJCgkJCQkJCQkgfQoJCQkJCQl9CgkJCQkJfQ=="}]]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634)
at org.elasticsearch.search.SearchService.createContext(SearchService.java:507)
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:480)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:252)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: [Error: unknown class or illegal statement: org.elasticsearch.common.mvel2.ParserContext#7915fb96]
[Near : {... doc.billingSequence.value as Integer ....}]
^
I have tried the given query and now it throws the below exception.
[DEBUG] 2015-10-26 10:33:01,912 org.elasticsearch.action.search.type - [Eson the Searcher] [owce_assets][4], node[aNnn7tvHRi6tCkOA-RU2nw], [P], s[STARTED]: Failed to execute [org.elasticsearch.action.search.SearchRequest#66accd25] lastShard [true]
org.elasticsearch.search.SearchParseException: [owce_assets][4]: from[0],size[100]: Parse Failure [Failed to parse source [{"from":0,"size":100,"aggregations_binary":"ewoiY291bnQiOiB7CiAgInRlcm1zIjogewogICAgInNjcmlwdCI6ICJkb2MuYmlsbGluZ1NlcXVlbmNlLnZhbHVlIGFzIEludGVnZXIiLAogICAgIm9yZGVyIjogewogICAgICAiX3Rlcm0iOiAiYXNjIgogICAgfSwKICAgICJ2YWx1ZV90eXBlIjogImludGVnZXIiLAogICAgInNpemUiOiAxMCwKICAgICJsYW5nIjogImdyb292eSIgICAgICAgICAgICAKICB9Cn0KfQ=="}]]
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:634)
at org.elasticsearch.search.SearchService.createContext(SearchService.java:507)
at org.elasticsearch.search.SearchService.createAndPutContext(SearchService.java:480)
at org.elasticsearch.search.SearchService.executeQueryPhase(SearchService.java:252)
at org.elasticsearch.search.action.SearchServiceTransportAction.sendExecuteQuery(SearchServiceTransportAction.java:202)
at org.elasticsearch.action.search.type.TransportSearchQueryThenFetchAction$AsyncAction.sendExecuteFirstPhase(TransportSearchQueryThenFetchAction.java:80)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:216)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction.performFirstPhase(TransportSearchTypeAction.java:203)
at org.elasticsearch.action.search.type.TransportSearchTypeAction$BaseAsyncAction$2.run(TransportSearchTypeAction.java:186)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:744)
Caused by: org.elasticsearch.ElasticsearchIllegalArgumentException: script_lang not supported [groovy]
at org.elasticsearch.script.ScriptService.compile(ScriptService.java:143)
at org.elasticsearch.script.ScriptService.search(ScriptService.java:163)
at
org.elasticsearch.search.aggregations.bucket.terms.TermsParser.parse(TermsParser.java:202)
at org.elasticsearch.search.aggregations.AggregatorParsers.parseAggregators(AggregatorParsers.java:114)
at org.elasticsearch.search.aggregations.AggregatorParsers.parseAggregators(AggregatorParsers.java:77)
at org.elasticsearch.search.aggregations.AggregationParseElement.parse(AggregationParseElement.java:60)
at org.elasticsearch.search.aggregations.AggregationBinaryParseElement.parse(AggregationBinaryParseElement.java:42)
at org.elasticsearch.search.SearchService.parseSource(SearchService.java:622)
... 11 more
The query I am passing as a parameter is::
{
"count": {
"terms": {
"script": "Integer.parseInt(doc.billingSequence.value)",
"order": {
"_term": "asc"
},
"value_type": "integer",
"size": 20
}
}
}
As my indexer contains 1, 2, 4, 6, 14 values in the billingSequence field. But after executing this query, I am getting 2, 4, 6 only in the result.
Exactly my code goes like below..
<mix:variable name="aggregations" as="xs:string">
<mix:option name="disable-expression-parsing" value="true"/>
{
"fatal_errors": {
"filter": {
"and":[
<!-- Filter by date (if given). Value should be send as "`DATE#" and the period should be in Filter data(<Filter name="field_name" value="`DATE#">1d</Filter>).-->
<mix:for-each select="//Date">
<mix:if test="$comma">,</mix:if>
{ "range": {
"<mix:value-of select='./#name'/>": {
"gte": "now-<mix:value-of select='./#value'/>",
"lt": "now"
}
}
}
<mix:global-variable name="comma" select="true()" as="xs:boolean"/>
</mix:for-each>
<!-- Filter all the not nulls or blanks in a field (if given). Value should be send as "`NOT_NULL#" (<Filter name="field_name" value="`NOT_NULL#"></Filter>).-->
<mix:for-each select="//NotNull">
<mix:if test="$comma">,</mix:if>
{
"not":
{
"term":{
"<mix:value-of select='./#name'/>":""
}
}
}
<mix:global-variable name="comma" select="true()" as="xs:boolean"/>
</mix:for-each>
<mix:for-each select="//FitlerTerm">
<mix:if test="$comma">,</mix:if>
{
"term":{
"<mix:value-of select='./#name'/>":"<mix:value-of select='./#value'/>"
}
}
<mix:global-variable name="comma" select="true()" as="xs:boolean"/>
</mix:for-each>
]
},
"aggs": {
"count": {
"terms": {
"script": "Integer.parseInt(doc.billingSequence.value)",
"order": {
"_term": "asc"
},
"value_type": "integer",
"size": 20
}
}
}
}
}
<mix:option name="disable-expression-parsing" value="false"/>
</mix:variable>
Here I am passing an aggregation with filter.. My filter is working fine and at the place of aggregation I am getting the issue. Please check and do the needful.
It seems that you're using mvel scripting. You simply need to specify that you want to use groovy instead.
{
"size": 0,
"aggs": {
"count": {
"terms": {
"script": "doc.billingSequence.value as Integer",
"order": {
"_term": "asc"
},
"value_type": "integer",
"size": 10,
"lang": "groovy" <--- add this line
}
}
}
}
Also make sure that you include this to your project dependencies as it's not included by default:
<dependency>
<groupId>org.codehaus.groovy</groupId>
<artifactId>groovy-all</artifactId>
<version>2.4.0</version>
<scope>compile</scope>
<optional>true</optional>
</dependency>
UPDATE
If you really cannot use Groovy, you can stick to MVEL and use MVEL type casting instead:
{
"size": 0,
"aggs": {
"count": {
"terms": {
"script": "Integer.parseInt(doc.billingSequence.value)", <--- change this line
"order": {
"_term": "asc"
},
"value_type": "integer",
"size": 10
}
}
}
}

Related Links

Removing From ElasticSearch by type last 7 day
Elasticsearch: accuracy on a filter aggregation
Wildcard query over _all field on Elasticsearch
Showing unmatched word in elasticsearch results
Elasticsearch: querying on both nested object properties and parent properties
elasticsearch first setup - create index
Using ElasticSearch as alternative data store with applications updating both the DB and ES(with the help of Kafka). Is this a good idea?
Where Elasticsearch store the data on Mac
Elastica - multiple bool queries - subqueries
How do i add boosting to the elasticsearch mapping in ONGR?
Is it possible to use doc_count as a term in elasticsearch?
Elastic search aggregation with sort
Elasticsearhc- how to do multi search request on Sense or Head plugin
How do I configure/initialize a custom Elasticsearch Tokenizer
Combining log entries with logstash
How to import CSV data using Logstash for field type completion in Elasticsearch suggester

Categories

HOME
opencart
systemd
website
websocket
header
azure-data-lake
tfs2010
mailing-list
stack-overflow
game-maker
business-intelligence
spring-jpa
sql-injection
squid
tomcat8
rpmbuild
scheduled-tasks
infrared
amazon-mws
angular2-highcharts
gitkraken
pst
sendmessage
spring-annotations
sparkle
bitbake
splunk-query
nic
payeezy
dreamfactory
pdfnet
ftp-server
variable-variables
imgur
flowchart
system-on-chip
code-push
adblock
realstudio
dredd
qvtkwidget
visual-format-language
1010
java-2d
logical
fuzzer
system.net.mail
wolfram-language
perforce-integrate
apache-pig-grunt
node-glob
gulp-uglify
management-studio-express
ioctl
astropy
http.client
auto-generate
extend
componentart
ampersand
clarion
azure-xplat-cli
tidyr
sunos
producer
visual-c++-2012
nidaqmx
nachos
.htpasswd
air-native-extension
gpu-programming
autonumber
saga
rautomation
delphi-xe4
days
itunes-sdk
contextswitchdeadlock
gprof
cgrectmake
onconfigurationchanged
chronometer
uitextfielddelegate
gwt-celltable
mvcrecaptcha
servercontrol
representation
digiflow
suphp
bass
eventaggregator
firephp
eai
html-generation
bindable-linq

Resources

Encrypt Message



code
soft
python
ios
c
html
jquery
cloud
mobile