[elasticsearch] how to rename an index in a cluster?

I need to rename several indexes in a cluster (their name must be changed, I cannot use aliases).

I saw that there are no supported ways to do that, the closest I found is to rename the directory of the index, I tried this in a cluster.

The cluster has 3 machines A, B and C and the shards are replicated on each of them. I shut down elasticsearch on A, renamed /var/lib/elasticsearch/security/nodes/0/indices/oldindexname to /var/lib/elasticsearch/security/nodes/0/indices/newindexname and restarted A.

The state of the cluster was yellow and elasticsearch was doing some magic to restore a correct state. After some time I ended up with

• oldindexname being available and fully replicated (recovered from B and C I guess)
• newindexname being available (I can search it) but the head plugin shows that its shards are in an "Unassigned" state and that they are grayed out (not replicated)

During the recovery security.log showed the following message:

[2015-02-20 11:02:33,461][INFO ][gateway.local.state.meta ] [A.example.com] dangled index directory name is [newindexname], state name is [oldindexname], renaming to directory name

While newindexname is searchable, it is certainly not in a normal state.

I rolled back to the previous state by deleting newindexname. The cluster is back to green without any "Unassigned" entries.

Given that, how can I rename oldindexname to newindexname in a cluster?

Note: the ultimate solution I have in mind is to scroll-copy oldindex into newindex and delete oldindex afterwards. This is going to take time so if there is a more direct solution it would be great.

As such there is no direct method to copy or rename index in ES (I did search extensively for my own project)

However a very easy option is to use a popular migration tool [Elastic-Exporter].

http://www.retailmenot.com/corp/eng/posts/2014/12/02/elasticsearch-cluster-migration/

[PS: this is not my blog, just stumbled upon and found it good]

Thereby you can copy index/type and then delete the old one.

As indicated in Elasticsearch reference for snapshot module,

The rename_pattern and rename_replacement options can be also used to rename index on restore using regular expression

You can use REINDEX to do that.

Reindex does not attempt to set up the destination index. It does not copy the settings of the source index. You should set up the destination index prior to running a _reindex action, including setting up mappings, shard counts, replicas, etc.

1. First copy the index to a new name

POST /_reindex
{
  "source": {
    "index": "twitter"
  },
  "dest": {
    "index": "new_twitter"
  }
}

2. Now delete the Index

DELETE /twitter

Just in case someone still needs it. The successful, not official, way to rename indexes are:

1. Close indexes that need to be renamed
2. Rename indexes' folders in all data directories of master and data nodes.
3. Reopen old closed indexes (I use kofp plugin). Old indexes will be reopened but stay unassigned. New indexes will appear in closed state
4. Reopen new indexes
5. Delete old indexes

If you happen to get this error "dangled index directory name is", remove index folder in all master nodes (not data nodes), and restart one of the data nodes.

Another different way to achieve the renaming or change the mappings for an index is to reindex using logstash. Here is a sample of the logstash 2.1 configuration:

input {
  elasticsearch {
   hosts => ["es01.example.com", "es02.example.com"]
   index => "old-index-name"
   size => 500
   scroll => "5m"
  }
}
filter {

 mutate {
  remove_field => [ "@version" ]
 }

 date {
   "match" => [ "custom_timestamp", "MM/dd/YYYY HH:mm:ss" ]
   target => "@timestamp"
 }

}
output {
 elasticsearch {
   hosts => ["es01.example.com", "es02.example.com" ]
   manage_template => false
   index => "new-index-name"
 }
}

For renaming your index you can use Elasticsearch Snapshot module.

First you have to take snapshot of your index.while restoring it you can rename your index.

    POST /_snapshot/my_backup/snapshot_1/_restore
    {
     "indices": "jal",
     "ignore_unavailable": "true",
     "include_global_state": false,
     "rename_pattern": "jal",
     "rename_replacement": "jal1"
     }

rename_replacement :-New indexname in which you want backup your data.

If you can't REINDEX a workaround is to use aliases. From the official documentation:

APIs in elasticsearch accept an index name when working against a specific index, and several indices when applicable. The index aliases API allow to alias an index with a name, with all APIs automatically converting the alias name to the actual index name. An alias can also be mapped to more than one index, and when specifying it, the alias will automatically expand to the aliases indices. An alias can also be associated with a filter that will automatically be applied when searching, and routing values. An alias cannot have the same name as an index.

Be aware that this solution does not work if you're using More Like This feature. https://github.com/elastic/elasticsearch/issues/16560