Elasticsearch 创建快照存储库抛出 RepositoryVerificationException

Question

我正在尝试为弹性搜索集群拍摄快照。设计如下。在 Docker 容器中，有 3 个 VM 分别运行 1 个主节点、1 个数据节点和 1 个客户端节点。每个 VM 都有一个附加的卷用于存储。因此，一个具有 3 个主节点、3 个客户端、3 个数据节点和 3 个卷的集群。

阅读文档后，我创建了一个单独的备份卷，并附加到其中一个虚拟机上。之后，我在所有 3 个虚拟机之间创建了一个 NFS，将数据保存在备份卷上，然后我修改了集群并将共享 NFS 目录作为卷挂载到集群中的所有节点

所以现在每个虚拟机都有以下内容：

VM1：

drwxr-xr-x  16 root   root     3560 Jul 24 10:30 dev
drwxr-xr-x   2 nobody nogroup  4096 Jul 24 11:49 elastic-backup
drwxr-xr-x  97 root   root     4096 Jul 24 14:04 etc
drwxr-xr-x   5 root   root     4096 Apr 27 12:53 home

虚拟机2：

drwxr-xr-x   2 root   root     4096 Jul 24 13:52 bin
drwxr-xr-x   3 root   root     4096 Jul 24 12:09 boot
drwxr-xr-x   5 root   root     4096 Jan 27 16:41 data
drwxr-xr-x  16 root   root     3580 Jul 24 11:48 dev
drwxr-xr-x   2 nobody nogroup  4096 Jul 24 11:49 elastic-backup

VM3：

drwxr-xr-x   3 root   root     4096 Jul 24 15:28 boot
drwxr-xr-x   5 root   root     4096 Jan 27 16:41 data
drwxr-xr-x  16 root   root     3560 Jul 24 10:30 dev
drwxr-xr-x   2 nobody nogroup  4096 Jul 24 15:34 elastic-backup

当我在其中创建文件时，我可以看到它、修改或其他任何操作，并且每个 VM 都可以看到该操作。

Elasticsearch docker 节点：

drwxr-xr-x 1 elasticsearch elasticsearch   4096 May 15  2018 config
drwxr-xr-x 4 elasticsearch elasticsearch   4096 Jul 23 12:15 data
drwxr-xr-x 2 elasticsearch elasticsearch   4096 Jul 24 15:08 elastic-backup

每个 docker elasticsearch 节点都挂载了相同的目录。我可以看到每个节点的所有文件。

问题是每当我尝试创建快照存储库时，我都会收到以下错误：

呼叫：

PUT /_snapshot/elastic-backup-1
{
  "type": "fs",
  "settings": {
    "location": "/usr/share/elasticsearch/elastic-backup"
  }
}

错误：

{
  "error": {
    "root_cause": [
      {
        "type": "repository_verification_exception",
        "reason": "[elastic-backup-1] [[some-id, 'RemoteTransportException[[master-2][VM2-ip][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[elastic-backup-1] a file written by master to the store [/usr/share/elasticsearch/elastic-backup] cannot be accessed on the node [{master-2}{some-id}{some-id}{VM2-ip}{VM2-ip}{zone=AZ2}]. This might indicate that the store [/usr/share/elasticsearch/elastic-backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [some-id, 'RemoteTransportException[[data-2][VM2-ip][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[elastic-backup-1] a file written by master to the store [/usr/share/elasticsearch/elastic-backup] cannot be accessed on the node [{data-2}{some-id}{some-id}{VM2-ip}{VM2-ip}{zone=AZ2}]. This might indicate that the store [/usr/share/elasticsearch/elastic-backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [some-id, 'RemoteTransportException[[data-1][VM1-ip][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[elastic-backup-1] a file written by master to the store [/usr/share/elasticsearch/elastic-backup] cannot be accessed on the node [{data-1}{some-id}{some-id}{VM1-ip}{VM1-ip}{zone=AZ1}]. This might indicate that the store [/usr/share/elasticsearch/elastic-backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [some-id, 'RemoteTransportException[[master-1][VM1-ip][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[elastic-backup-1] a file written by master to the store [/usr/share/elasticsearch/elastic-backup] cannot be accessed on the node [{master-1}{some-id}{some-id}{VM1-ip}{VM1-ip}{zone=AZ1}]. This might indicate that the store [/usr/share/elasticsearch/elastic-backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];'], [some-id, 'RemoteTransportException[[data-3][VM3-ip][internal:admin/repository/verify]]; nested: RepositoryVerificationException[[elastic-backup-1] a file written by master to the store [/usr/share/elasticsearch/elastic-backup] cannot be accessed on the node [{data-3}{some-id}{some-id}{VM3-ip}{VM3-ip}{zone=AZ1}]. This might indicate that the store [/usr/share/elasticsearch/elastic-backup] is not shared between this node and the master node or that permissions on the store don't allow reading files written by the master node];']]"
      }
etc ..

我做错了什么？如何解决这个问题

Answer 1

如Christian_Dahlqvist 所述，您必须提供共享文件系统。

您需要有一个共享卷，例如NFS 卷，位于所有节点都可以访问的存储库路径后面。这意味着如果 node1 写入了一个文件，那么它对节点 2 和节点 3 都是可见的。因此，本地文件系统中的目录将不起作用，即使所有机器上的路径都相同。