how to copy one index documents to other index in elasticsearch?

Question

I created a new index with mappings. 500 000 documents are stored in it.

I want to change the mapping of the index, but it is not possible in elastic search.so I created another index with new new mappings and now I am trying to copy the documents from old index to new index.

I am using scan and scroll type to retrieve the documents from old index and copying to new index.for copying its taking more time and the system is slowing down.

Below is the code that I am using.

$client= app('elastic_search');
    $params = [
        "search_type" => "scan",    // use search_type=scan
        "scroll" => "30s",          // how long between scroll requests. should be small!
        "size" => 500000,               // how many results *per shard* you want back
        "index" => "admin_logs422",
        "body" => [
            "query" => [
                "match_all" => []
            ]
        ]
    ];

    $docs = $client->search($params);   // Execute the search

    $scroll_id = $docs['_scroll_id'];   


    while (rue) {

        // Execute a Scroll request
        $response = $client->scroll([
                "scroll_id" => $scroll_id,  //...using our previously obtained _scroll_id
                "scroll" => "500s"           // and the same timeout window
            ]
        );

        if (count($response['hits']['hits']) > 0) {
            foreach($response['hits']['hits'] as $s)
            {

                $params =
                    [
                        'index' => 'admin_logs421',
                        'type' => 'admin_type421',
                        'id'=> $s['_id'],
                        'client' => [
                            'ignore' => [400, 404],
                            'verbose' => true,
                            'timeout' => 10,
                            'connect_timeout' => 10
                        ],
                        'body' => $s['_source']
                    ];

                $response = app('elastic_search')->create($params);

            }


            $scroll_id = $response['_scroll_id'];
        } else {
            // No results, scroll cursor is empty.  You've exported all the data
            return response("completed");
        }
    }

Answer 1

You should not have to code stuff like that. There are some excellent tools around to do it for you.

Just look at Taskrabbit's elasticdump utility which does exactly what you want.

elasticdump 
  --input=http://localhost:9200/source_index 
  --output=http://localhost:9200/target_index 
  --type=data

Or you can also very easily leverage Logstash as is shown in this other answer

Finally, since you're using Python, you can also use the elasticsearch-py reindex utility