Styjun

I'm trying to run a boolean query against an elastic-search cluster with a basic shakespeare dataset. I've crosschecked many resources and everything seems correct, but when i run it, the speech_number scores OR operation doesn't work as expected.

I've looked through various tutorials and documentation on elasticsearch boolean queries and I still haven't found out why the logic doesnt work as expected.

"query": 
 "bool": 
 "must": [
 
 "match": "play_name": "Henry IV"
 ,
 
 "bool": 
 "should": [
 "range": "speech_number": "lte": 50,
 "range": "speech_number": "gte": 4
 ]
 
 
 ]
 
 
 }

A sample of the shakespear.json file which im running the queries against is as follows:

"line_id":1658,"play_name":"Henry IV","speech_number":26,"line_number":"3.1.108","speaker":"MORTIMER","text_entry":"Yea, but"
"index":"_index":"shakespeare","_type":"line","_id":1658
"line_id":1659,"play_name":"Henry IV","speech_number":26,"line_number":"3.1.109","speaker":"MORTIMER","text_entry":"Mark how he bears his course, and runs me up"
"index":"_index":"shakespeare","_type":"line","_id":1659
"line_id":1660,"play_name":"Henry IV","speech_number":26,"line_number":"3.1.110","speaker":"MORTIMER","text_entry":"With like advantage on the other side;"
"index":"_index":"shakespeare","_type":"line","_id":1660
"line_id":1661,"play_name":"Henry IV","speech_number":26,"line_number":"3.1.111","speaker":"MORTIMER","text_entry":"Gelding the opposed continent as much"
"index":"_index":"shakespeare","_type":"line","_id":1661
"line_id":1662,"play_name":"Henry IV","speech_number":26,"line_number":"3.1.112","speaker":"MORTIMER","text_entry":"As on the other side it takes from you."
"index":"_index":"shakespeare","_type":"line","_id":1662
"line_id":1663,"play_name":"Henry IV","speech_number":27,"line_number":"3.1.113","speaker":"EARL OF WORCESTER","text_entry":"Yea, but a little charge will trench him here"
"index":"_index":"shakespeare","_type":"line","_id":1663
"line_id":1664,"play_name":"Henry IV","speech_number":27,"line_number":"3.1.114","speaker":"EARL OF WORCESTER","text_entry":"And on this north side win this cape of land;"
"index":"_index":"shakespeare","_type":"line","_id":1664
"line_id":1665,"play_name":"Henry IV","speech_number":27,"line_number":"3.1.115","speaker":"EARL OF WORCESTER","text_entry":"And then he runs straight and even."
"index":"_index":"shakespeare","_type":"line","_id":1665
"line_id":1666,"play_name":"Henry IV","speech_number":28,"line_number":"3.1.116","speaker":"HOTSPUR","text_entry":"Ill have it so: a little charge will do it."
"index":"_index":"shakespeare","_type":"line","_id":1666
"line_id":1667,"play_name":"Henry IV","speech_number":29,"line_number":"3.1.117","speaker":"GLENDOWER","text_entry":"Ill not have it alterd."
"index":"_index":"shakespeare","_type":"line","_id":1667
"line_id":1668,"play_name":"Henry IV","speech_number":30,"line_number":"3.1.118","speaker":"HOTSPUR","text_entry":"Will not you?"
"index":"_index":"shakespeare","_type":"line","_id":1668
"line_id":1669,"play_name":"Henry IV","speech_number":31,"line_number":"3.1.119","speaker":"GLENDOWER","text_entry":"No, nor you shall not."
"index":"_index":"shakespeare","_type":"line","_id":1669
"line_id":1670,"play_name":"Henry IV","speech_number":32,"line_number":"3.1.120","speaker":"HOTSPUR","text_entry":"Who shall say me nay?"
"index":"_index":"shakespeare","_type":"line","_id":1670
"line_id":1671,"play_name":"Henry IV","speech_number":33,"line_number":"3.1.121","speaker":"GLENDOWER","text_entry":"Why, that will I."

The expected result should be: play_name AND (speech_number <= 50 OR speech_number >=4)
What i get is: play_name AND (speech_number <=50 AND speech_number >= 4)

You are correct, the query is doing:

• Must match:
  1) Any of the words from "Henry IV" [1 see below]
  2) A speech_number <= 50 [OR] speech_number >=4

What Elasticsearch is also doing, is assigning a score: So everything inside the must should match and then everything inside should queries is boosting results (at least one of a should has to match)

To boost a higher speech_number more (don't do this) you could have more matching should statements:

 "query": 
 "bool": 
 "must": [
 
 "match": 
 "play_name": "Henry IV"
 
 ,

 
 "bool": 
 "should": [
 
 "range": 
 "speech_number": "lte": 50 
 
 ,

 
 "range": 
 "speech_number": "lte": 40 
 
 ,

 
 "range": 
 "speech_number": "lte": 30 
 
 ,

 ...
 ]
 
 
 ]
 
 

So a part of the problem could be lte: 50 will allow <4, and gte:4 >50. But I don't see any outsiders in your results. If it's the ordering. The range also allows for a boost (https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-range-query.html), so instead of having to write multiple ranges you could have:

 "query": 
 "bool": 
 "must": [
 
 "match": 
 "play_name": "Henry IV",
 "operator": "and
 
 ,

 
 "bool": 
 "should": [
 
 "range": 
 "speech_number": 
 "gte": 25,
 "lte": 50,

 "boost": 3
 
 
 ,

 
 "range": 
 "speech_number": 
 "gte": 4,
 "lte": 50
 
 
 
 ]
 
 
 ]
 
 

[1*] match defaults to OR: https://www.elastic.co/guide/en/elasticsearch/reference/current/query-dsl-match-query.html if your data is structured, a Term or adding the operator and to it is more what you want. Not the problem of your question :)