[ruby] Setting Elastic search limit to "unlimited"

How can i get all the results from elastic search as the results only display limit to 10 only. ihave got a query like:

@data = Athlete.search :load => true do
          size 15
          query do
            boolean do
              must { string q, {:fields => ["name", "other_names", "nickname", "short_name"], :phrase_slop => 5} }
              unless conditions.blank?
                conditions.each do |condition|
                  must { eval(condition) }
                end
              end
              unless excludes.blank?
                excludes.each do |exclude|
                  must_not { eval(exclude) }
                end
              end
            end
          end
          sort do
            by '_score', "desc"
          end
        end

i have set the limit to 15 but i wan't to make it unlimited so that i can get all the data I can't set the limit as my data keeps on changing and i want to get all the data.

Another approach is to first do a searchType: 'count', then and then do a normal search with size set to results.count.

The advantage here is it avoids depending on a magic number for UPPER_BOUND as suggested in this similar SO question, and avoids the extra overhead of building too large of a priority queue that Shay Banon describes here. It also lets you keep your results sorted, unlike scan.

The biggest disadvantage is that it requires two requests. Depending on your circumstance, this may be acceptable.

From the docs, "Note that from + size can not be more than the index.max_result_window index setting which defaults to 10,000". So my admittedly very ad-hoc solution is to just pass size: 10000 or 10,000 minus from if I use the from argument.

Note that following Matt's comment below, the proper way to do this if you have a larger amount of documents is to use the scroll api. I have used this successfully, but only with the python interface.

use the scan method e.g.

 curl -XGET 'localhost:9200/_search?search_type=scan&scroll=10m&size=50' -d '
 {
    "query" : {
       "match_all" : {}
     }
 }

see here