elasticsearch


Why is my query ignoring my filter aggregation?


Preface
I have 4 days experience of Elasticsearch 1.7.2.
Setup
I have a collection of documents, each document is a User. The User has a number of Answers which is linked through UserAnswers. Which gives a document reference of user_answers.answer[]. Where the answers array is an array of objects.
The user_answers.answer[].correct is a boolean field which tells me if the answer given by the user is correct or not.
Objective
I would like to list the users and also display the total number of correct and incorrect answers they have.
Approach
So far I have tried a number of different approaches and the one I'll include here is as close as I've got so far in 1.5 days of trying.
Use a terms aggregation to create a bucket for each User by username.
Filter each bucket to leave only correct or incorrect answers.
Count the number of filtered answers.
Query
{
"size": 0,
"filter": {
"bool": {
"must_not": {
// Remove users who already have this award
"term": {"awards_users.award_id": 2}
}
}
},
"aggs": {
"users": {
"terms": {"field": "username"},
"aggs": {
"correct": {
"filter": {
"term": {"user_answers.answer.correct": true}
},
"aggs": {
"count": {
"value_count": {
"field": "user_answers.answer.id"
}
}
}
},
// Same for incorrect, but inverted correct value
}
}
}
}
Sample response
{
"key": "neon1024",
"doc_count": 1,
"correct": {
"doc_count": 1,
"count": {
"value": 7 // Expected 1 correct & 6 incorrect
}
}
},
This is the record which I am testing against, and I am expecting that 1 is returned instead of 7. There are 7 answers in total, 6 incorrect and 1 correct. This I have verified in my document index.
The problem
For some reason the actual filter seems to be being ignored, and leaving all possible related answers in the bucket. Hence the aggregation is seeing them all, rather than showing the expected value.
Question
How can I use an aggregation to segregate my counts based on the value of the related answers values?
Thanks for reading my long question!
As suggested, you probably have your answers mapped as object, while you should be using nested type.
Using nested type, elasticsearch will store your answers as individual documents linked to the root one and will let you do expected aggregations on them. You'll have to use nested type aggregation in your query to achieve that.
So I'd say it would be best to map your document like this:
PUT /test
{
"mappings" : {
"your_type" : {
"properties" : {
"username" : {
"type" : "string",
"index" : "not_analyzed"
},
"user_answers" : {
"type" : "nested",
"properties" : {
"id" : {
"type" : "integer"
},
"answer" : {
"type" : "string"
},
"correct" : {
"type" : "boolean"
}
}
}
}
}
}
}
Test document:
PUT /test/your_type/1
{
"username": "neon1024",
"user_answers": [
{
"id": 1,
"answer": "answer1",
"correct": true
},
{
"id": 2,
"answer": "answer2",
"correct": true
},
{
"id": 3,
"answer": "answer3",
"correct": false
}
]
}
Query:
POST /test/_search?search_type=count
{
"aggs": {
"users": {
"terms": {
"field": "username"
},
"aggs": {
"DiveIn": {
"nested": {
"path": "user_answers"
},
"aggs": {
"CorrectVsIncorrect": {
"terms": {
"field": "user_answers.correct",
"size": 2
}
}
}
}
}
}
}
}
And Final result:
{
"took": 6,
"timed_out": false,
"_shards": {
"total": 5,
"successful": 5,
"failed": 0
},
"hits": {
"total": 1,
"max_score": 0,
"hits": []
},
"aggregations": {
"users": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "neon1024",
"doc_count": 1,
"DiveIn": {
"doc_count": 3,
"CorrectVsIncorrect": {
"doc_count_error_upper_bound": 0,
"sum_other_doc_count": 0,
"buckets": [
{
"key": "T",
"doc_count": 2
},
{
"key": "F",
"doc_count": 1
}
]
}
}
}
]
}
}
}
Where "key": "T" represents correct answers and "doc_count": 2 represents amount of them.

Related Links

Where Elasticsearch store the data on Mac
Elastica - multiple bool queries - subqueries
How do i add boosting to the elasticsearch mapping in ONGR?
Is it possible to use doc_count as a term in elasticsearch?
Elastic search aggregation with sort
Elasticsearhc- how to do multi search request on Sense or Head plugin
How do I configure/initialize a custom Elasticsearch Tokenizer
Combining log entries with logstash
How to import CSV data using Logstash for field type completion in Elasticsearch suggester
How can I interpret the result of percentiles aggregation type
elastic search select fields to return in aggregation
How to Focus Search On One Part of a Document
How to calculate cooccurrences on a set of texts with Elasticsearch
elasticsearch edge_ngrams analyzer is not found
Double wildcard in query causes weird highlighting for plain/fast vectors elasticsearch highlighters
Get Elasticsearch response in DataTable in C#.Net

Categories

HOME
openshift
azure-functions
keyboard
mpdf
verification
apple-push-notifications
vagrant
opencv-python
selenium-builder
kivy
embedded-resource
odata
google-webmaster-tools
fogbugz
moodle-api
visualforce
clone
sympy
spring-security-oauth2
fbloginview
xbox360
synthesis
google-tasks-api
asciimath
rpmbuild
boomi
home
substring
password-generator
ipa
ibm-connections
javax.persistence
google-play-developer-api
toad
twilio-php
python-sounddevice
best-buy-api
microkernel
stat
grunt-contrib-watch
opal-framework
asihttprequest
spongycastle
cyanogenmod
deployd
uiautomatorviewer
knockout-2.0
visual-format-language
sequence-diagram
mifos
ipywidgets
s-function
xmlbeans
ndk-build
yii2-model
highlighting
adjacency-list
swiffy
accessibilityservice
trust
cortana
diawi
json-spirit
modern.ie
google-gdk
encryption-symmetric
iplimage
oracle-adf-mobile
category-theory
treeline
distributed-r
radius-protocol
armv6
visual-c++-2005
infinite
non-ascii-characters
insertion
bcdedit
asp.net-mvc-3-areas
system-requirements
story
getproperty
viewpagerindicator
websphere-6.1
background-repeat
expression-web
space-partitioning
nsconnection
eaccelerator
ccl
icicles
powergui
xap
iphone-sdk-3.1.3
synthesizer

Resources

Encrypt Message



code
soft
python
ios
c
html
jquery
cloud
mobile