Make elasticsearch only return certain fields

Question

I m using elasticsearch to index my documents    Is it possible to instruct it to only return particular fields instead of the entire json document it has stored

User · Answer

if you know sql  please write a query to get the code s value for example sql query equivalent and elasticsearch query POST   sql translate         quot query quot    quot select name surname from users quot     result is  be carefull look at the includes key      quot size quot    1000     quot  source quot           quot includes quot             quot name quot          quot surname quot              quot excludes quot                quot sort quot                   quot  doc quot               quot order quot     quot asc quot

User · Answer

In Elasticsearch 5 x the above mentioned approach is deprecated  You can use the  source approach  but but in certain situations it can make sense to store a field  For instance  if you have a document with a title  a date  and a very large content field  you may want to retrieve just the title and the date without having to extract those fields from a large  source field   In this case  you d use           size    INT NUM OF DOCS TO RETURN      stored fields             doc headline          doc text          doc timestamp utc            query             bool                must                   term                      doc topic   news on things                                      filter                   range                      doc timestamp utc                         gte  1451606400000                     lt  1483228800000                     format   epoch millis                                                              aggs                See the documentation on how to index stored fields  Always happy for an Upvote

User · Answer

A REST API GET request could be made with   source  parameter   Example Request  http   localhost 9200 opt pr  search q SYMBOL ITC AND OPTION TYPE CE AND TRADE DATE 2017-02-10 AND EXPIRY DATE 2017-02-23 amp  source STRIKE PRICE   Response     took   59   timed out   false    shards          total   5       successful   5       failed   0     hits          total   104       max score   7 3908954       hits                             index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLc                 score   7 3908954                source                      STRIKE PRICE   160                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLh                 score   7 3908954                source                      STRIKE PRICE   185                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLi                 score   7 3908954                source                      STRIKE PRICE   190                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLm                 score   7 3908954                source                      STRIKE PRICE   210                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLp                 score   7 3908954                source                      STRIKE PRICE   225                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLr                 score   7 3908954                source                      STRIKE PRICE   235                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLw                 score   7 3908954                source                      STRIKE PRICE   260                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uL5                 score   7 3908954                source                      STRIKE PRICE   305                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLd                 score   7 381078                source                      STRIKE PRICE   165                                                  index    opt pr                 type    opt pr r                 id    AV3K4QTgNHl15Mv30uLy                 score   7 381078                source                      STRIKE PRICE   270

User · Answer

Here is another solution  now using a match expression Source filtering allows to control how the  source field is returned with every hit  Tested with Elastiscsearch version 5 5 The keyword includes defines the specifics fields  GET  my indice my indice type  search      quot  source quot          quot includes quot            quot my especific field quot                quot query quot          quot bool quot            quot must quot                          quot match quot                  quot  id quot    quot  my id here without percent  quot

User · Answer

here you can specify whichever field you want in your output and also which you don t       POST index name  search                quot  source quot                  quot includes quot      quot field name quot    quot field name quot                  quot excludes quot      quot field name quot                        quot query quot                   quot match quot       quot field name quot     quot value quot

User · Answer

response filtering     All REST APIs accept a filter path parameter that can be used to   reduce the response returned by elasticsearch  This parameter takes a   comma separated list of filters expressed with the dot notation    https   stackoverflow com a 35647027 844700

User · Answer

For example  you have a doc with three fields   PUT movie  doc 1      name   The Lion King      language   English      score   9 3      If you want to return name and score you can use the following command   GET movie  doc 1  source includes name score   If you want to get some fields which match a pattern   GET movie  doc 1  source includes  re   Maybe exclude some fields   GET movie  doc 1  source excludes score

User · Answer

Yep  Use a better option source filter   If you re searching with JSON it ll look something like this         quot  source quot     quot user quot    quot message quot              quot query quot             quot size quot          In ES 2 4 and earlier  you could also use the fields option to the search API         quot fields quot     quot user quot    quot message quot              quot query quot             quot size quot          This is deprecated in ES 5    And source filters are more powerful anyway

User · Answer

I found the docs for the get api to be helpful - especially the two sections  Source filtering and Fields  https   www elastic co guide en elasticsearch reference 7 3 docs-get html get-source-filtering  They state about source filtering      If you only need one or two fields from the complete  source  you can   use the  source include  amp   source exclude parameters to include or   filter out that parts you need  This can be especially helpful with   large documents where partial retrieval can save on network overhead   Which fitted my use case perfectly  I ended up simply filtering the source like so  using the shorthand            source     field x         field y         query                                 FYI  they state in the docs about the fields parameter      The get operation allows specifying a set of stored fields that will   be returned by passing the fields parameter    It seems to cater for fields that have been specifically stored  where it places each field in an array  If the specified fields haven t been stored it will fetch each one from the  source  which could result in  slower  retrievals  I also had trouble trying to get it to return fields of type object   So in summary  you have two options  either though source filtering or  stored  fields

User · Answer

There are several methods that can be useful to achieve field-specific results  One can be through the source method  And another method that can also be useful to receive cleaner and more summarized answers according to our interests is filter path  Document Json   quot hits quot               quot  index quot     quot xxxxxx quot        quot  type quot     quot  doc quot        quot  id quot     quot xxxxxx quot        quot  score quot    xxxxxx       quot  source quot             quot year quot    2020         quot created at quot     quot 2020-01-29 quot          quot url quot     quot www github com mbarr0987 quot          quot name quot   quot github quot             Query  GET bot1  search filter path hits hits  source url      quot query quot          quot bool quot            quot must quot               quot term quot     quot name keyword quot   quot github quot                           Output       quot hits quot           quot hits quot                       quot  source quot                 quot url quot     quot www github com mbarr0987 quot

User · Answer

In java you can use setFetchSource like this     client prepareSearch index  setTypes type               setFetchSource new String      field1    field2     null

User · Answer

For the ES versions 5 X and above you can a ES query something like this      GET                       quot  source quot              quot includes quot      quot FIELD1 quot    quot FIELD2 quot    quot FIELD3 quot       quot

User · Answer

Yes by using source filter you can accomplish this  here is the doc source-filtering  Example Request  POST index name  search         source    field1   filed2              Output will be       took   57     timed out   false      shards          total   5       successful   5       skipped   0       failed   0         hits          total   1       max score   1       hits                       index    index name             type    index1             id    1             score   1            source                field1    a              field2    b                                  field1    c              field2    d

[elasticsearch] Make elasticsearch only return certain fields?

Examples related to elasticsearch