How can i get all the results from elastic search as the results only display limit to 10 only. ihave got a query like:
@data = Athlete.search :load => true do
size 15
query do
boolean do
must { string q, {:fields => ["name", "other_names", "nickname", "short_name"], :phrase_slop => 5} }
unless conditions.blank?
conditions.each do |condition|
must { eval(condition) }
end
end
unless excludes.blank?
excludes.each do |exclude|
must_not { eval(exclude) }
end
end
end
end
sort do
by '_score', "desc"
end
end
i have set the limit to 15 but i wan't to make it unlimited so that i can get all the data I can't set the limit as my data keeps on changing and i want to get all the data.
This question is related to
ruby
elasticsearch
use the scan method e.g.
curl -XGET 'localhost:9200/_search?search_type=scan&scroll=10m&size=50' -d '
{
"query" : {
"match_all" : {}
}
}
see here
Another approach is to first do a searchType: 'count'
, then and then do a normal search with size
set to results.count
.
The advantage here is it avoids depending on a magic number for UPPER_BOUND
as suggested in this similar SO question, and avoids the extra overhead of building too large of a priority queue that Shay Banon describes here. It also lets you keep your results sorted, unlike scan
.
The biggest disadvantage is that it requires two requests. Depending on your circumstance, this may be acceptable.
From the docs, "Note that from + size
can not be more than the index.max_result_window
index setting which defaults to 10,000". So my admittedly very ad-hoc solution is to just pass size: 10000
or 10,000 minus from if I use the from
argument.
Note that following Matt's comment below, the proper way to do this if you have a larger amount of documents is to use the scroll api. I have used this successfully, but only with the python interface.
Source: Stackoverflow.com