how to rename an index in a cluster

Question

I need to rename several indexes in a cluster  their name must be changed  I cannot use aliases    I saw that there are no supported ways to do that  the closest I found is to rename the directory of the index  I tried this in a cluster   The cluster has 3 machines A  B and C and the shards are replicated on each of them  I shut down elasticsearch on A  renamed  var lib elasticsearch security nodes 0 indices oldindexname to  var lib elasticsearch security nodes 0 indices newindexname and restarted A   The state of the cluster was yellow and elasticsearch was doing some magic to restore a correct state  After some time I ended up with   oldindexname being available and fully replicated  recovered from B and C I guess   newindexname being available  I can search it  but the head plugin shows that its shards are in an  Unassigned  state and that they are grayed out  not replicated    During the recovery security log showed the following message    2015-02-20 11 02 33 461  INFO   gateway local state meta    A example com  dangled index directory name is  newindexname   state name is  oldindexname   renaming to directory name   While newindexname is searchable  it is certainly not in a normal state   I rolled back to the previous state by deleting newindexname  The cluster is back to green without any  Unassigned  entries   Given that  how can I rename oldindexname to newindexname in a cluster   Note  the ultimate solution I have in mind is to scroll-copy oldindex into newindex and delete oldindex afterwards  This is going to take time so if there is a more direct solution it would be great

User · Accepted Answer

Starting with ElasticSearch 7 4  the best method to rename an index is to copy the index using the newly introduced Clone Index API  then to delete the original index using the Delete Index API   The main advantage of the Clone Index API over the use of the Snapshot API or the Reindex API for the same purpose is speed  since the Clone Index API hardlinks segments from the source index to the target index  without reprocessing any of its content  on filesystems that support hardlinks  obviously  otherwise  files are copied at the file system level  which is still much more efficient that the alternatives   Clone Index also guarantee that the target index is identical in every point to the source index  that is  there is no need to manually copy settings and mappings  contrary to the Reindex approach   and doesn t require a local snapshot directory be configured   Side note  even though this procedure is much faster than previous solutions  it still implies down time  There are real use cases that justify renaming indices  for example  as a step in a split  shrink or backup workflow   but renaming indices should not be part of day-to-day operations  If your workflow requires frequent index renaming  then you should consider using Indices Aliases instead   Here is an example of a complete sequence of operations to rename index source index to target index  It can be executed using some ElasticSearch specific console  such as the one integrated in Kibana  See this gist for an alternative version of this example  using curl instead of an Elastic Search console     Make sure the source index is actually open POST  source index  open    Put the source index in read-only mode PUT  source index  settings      settings          index blocks write    true           Clone the source index to the target name  and set the target to read-write mode POST  source index  clone target index      settings          index blocks write   null           Wait until the target index is green    it should usually be fast  assuming your filesystem supports hard links   GET   cluster health target index wait for status green amp timeout 30s    If it appears to be taking too much time for the cluster to get back to green    the following requests might help you identify eventual outstanding issues  if any  GET   cat indices target index GET   cat recovery target index GET   cluster allocation explain    Delete the source index DELETE  source index

User · Answer

Another different way to achieve the renaming or change the mappings for an index is to reindex using logstash  Here is a sample of the logstash 2 1 configuration   input     elasticsearch      hosts   gt    es01 example com    es02 example com      index   gt   old-index-name     size   gt  500    scroll   gt   5m        filter     mutate     remove field   gt      version         date       match    gt     custom timestamp    MM dd YYYY HH mm ss       target   gt    timestamp        output    elasticsearch      hosts   gt    es01 example com    es02 example com       manage template   gt  false    index   gt   new-index-name

User · Answer

For renaming your index you can use Elasticsearch Snapshot module   First you have to take snapshot of your index while restoring it you can  rename your index       POST   snapshot my backup snapshot 1  restore             indices    jal         ignore unavailable    true         include global state   false        rename pattern    jal         rename replacement    jal1           rename replacement  -New indexname in which you want backup your data

User · Answer

As indicated in Elasticsearch reference for snapshot module       The rename pattern and rename replacement options can be also used to rename index on restore using regular expression

User · Answer

If you can t REINDEX a workaround is to use aliases  From the official documentation    APIs in elasticsearch accept an index name when working against a specific index  and several indices when applicable  The index aliases API allow to alias an index with a name  with all APIs automatically converting the alias name to the actual index name  An alias can also be mapped to more than one index  and when specifying it  the alias will automatically expand to the aliases indices  An alias can also be associated with a filter that will automatically be applied when searching  and routing values  An alias cannot have the same name as an index   Be aware that this solution does not work if you re using More Like This feature  https   github com elastic elasticsearch issues 16560

User · Answer

As such there is no direct method to copy or rename index in ES  I did search extensively for my own project   However a very easy option is to use a popular migration tool  Elastic-Exporter     http   www retailmenot com corp eng posts 2014 12 02 elasticsearch-cluster-migration    PS  this is not my blog  just stumbled upon and found it good   Thereby you can copy index type and then delete the old one

User · Answer

Just in case someone still needs it  The successful  not official  way to rename indexes are    Close indexes that need to be renamed Rename indexes  folders in all data directories of master and data nodes  Reopen old closed indexes  I use kofp plugin   Old indexes will be reopened but stay unassigned  New indexes will appear in closed state Reopen new indexes Delete old indexes   If you happen to get this error  dangled index directory name is   remove index folder in all master nodes  not data nodes   and restart one of the data nodes

User · Answer

You can use REINDEX to do that      Reindex does not attempt to set up the destination index  It does not   copy the settings of the source index  You should set up the   destination index prior to running a  reindex action  including   setting up mappings  shard counts  replicas  etc     First copy the index to a new name   POST   reindex      source          index    twitter          dest          index    new twitter           Now delete the Index   DELETE  twitter

[elasticsearch] how to rename an index in a cluster?

Examples related to elasticsearch