elasticsearch 7 - Problem on restore snapshot of "single node cluster" to "multi node cluster"

Question

I have a single node cluster in my local windows Elasticsearch database, I make a snapshot of my index successfully. I have a 3 nodes cluster in my VMware, which each node(server) is Ubuntu. While I was succeeded on restoring my snap shot to a single node cluster, Now I have problem on restoring my special Index to my multi node cluster. I registered a folder named "snapshots" as my repository in my local Elasticsearch-7.10-win64 and I took some snapshots from my Index in that folder:

PUT /_snapshot/my_fs_backup
{
  "type": "fs",
  "settings": {
    "location": "D:\snapshots",
    "compress": true
  }
}   
PUT /_snapshot/my_fs_backup/snapshot_1? wait_for_completion = true
   {
      "indices": "suggestionindex",
      "ignore_unavailable": true,
      "include_global_state": false,
      "metadata": {
        "taken_by": "Ehsan",
        "taken_because": "Testing"
      }
}

I'm done in windows, now let's go to 3 Ubuntu servers. I created a shared NFS folder named snapshots on all 3 nodes as described here: Elasticsearch: Snapshot Backups on a Shared NFS I registered my repository in my 3 nodes cluster truly as below :

PUT /_snapshot/my_fs_backup
{
  "type": "fs",
  "settings": {
    "location": "/var/lib/elasticsearch/snapshots",
    "compress": true
  }
}

Now it's time to restore my snapshot_1 to my 3 node cluster. So I copied the whole contents inside my local snapshots folder to the shared address /var/lib/elasticsearch/snapshots. Now Time to restore the snapshot_1:

POST /_snapshot/my_fs_backup/snapshot_1/_restore?wait_for_completion=true
{
  "indices": "suggestionindex",
  "ignore_unavailable": true,
  "include_global_state": false,
  "include_aliases": false
}

and the response is : (Dev Tools in Kibana)

{
  "snapshot" : {
    "snapshot" : "snapshot_1",
    "indices" : [
      "suggestionindex"
    ],
    "shards" : {
      "total" : 1,
      "failed" : 1,
      "successful" : 0
    }
  }
}

I do not know what's wrong but maybe these info could be useful :

POST /_snapshot/my_fs_backup/_verify results in:

{
  "nodes" : {
    "-S8rY_jOQMCvbAXvTRKQDw" : {
      "name" : "node-1"
    },
    "y4ZQBhkCTV6jcARKjvvh5A" : {
      "name" : "node-3"
    },
    "qlNcOrrJTtmvOOs4gCHgTw" : {
      "name" : "node-2"
    }
  }
}

POST /_snapshot/my_fs_backup/all results in:

{
  "error" : {
    "root_cause" : [
      {
        "type" : "snapshot_exception",
        "reason" : "[my_fs_backup:all/4afsnEkiSd6MErwY_V-9GQ] Indices don't have primary shards [suggestionindex]"
      }
    ],
    "type" : "snapshot_exception",
    "reason" : "[my_fs_backup:all/4afsnEkiSd6MErwY_V-9GQ] Indices don't have primary shards [suggestionindex]"
  },
  "status" : 500
}

Get /_cat/shards?v results in:

suggestionindex 0     r      UNASSIGNED                             
suggestionindex 0     p      UNASSIGNED                             
.kibana         0     r      STARTED      21 88.4kb 192.168.254.136 node-3
.kibana         0     p      STARTED      21 88.5kb 192.168.254.135 node-2

I created my suggestionIndex in my 3 nodes cluster with the same code as my local windows 10, with one primary shard on node-2 and replica on node-3, then I closed my suggestionindex and tried again to restore the snapshot, but still the same problem that it won't be assigned !

question from:https://stackoverflow.com/questions/65835283/problem-on-restore-snapshot-of-single-node-cluster-to-multi-node-cluster

Answer 1

You can get UNASSIGNED shards reason from command : curl localhost:9200/_cat/shards?h=i,s,pr,st,n,ur,ua,uf,ud | grep UNASSIGNED

Please share the output above command ?