Minikube：mount(2) 系统调用失败：连接被拒绝答案

【问题标题】：Minikube: mount(2) system call failed: Connection refusedMinikube：mount(2) 系统调用失败：连接被拒绝
【发布时间】：2019-12-01 22:59:34
【问题描述】：

尝试将本地驱动器安装到我的 minikube 主机（似乎是这个 thread 的欺骗，但没有提供解决方案......）

使用：

OSX 10.14.3 和 minikube（使用 HyperVisor）

$ minikube mount --ip 192.168.64.5 --v=7 ~/Documents/projects/docker_storage/tf:/mnt/vda1/data/tf
Found binary path at /usr/local/bin/docker-machine-driver-hyperkit
Launching plugin server for driver hyperkit
Plugin server listening at address 127.0.0.1:58272
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .DriverName
????  Mounting host path /Users/XXXXXX/Documents/projects/docker_storage/tf into VM as /mnt/vda1/data/tf ...
????  Mount options:
    ▪ Type:     9p
    ▪ UID:      docker
    ▪ GID:      docker
    ▪ Version:  9p2000.L
    ▪ MSize:    262144
    ▪ Mode:     755 (-rwxr-xr-x)
    ▪ Options:  map[]
(minikube) Calling .GetSSHHostname
(minikube) Calling .GetSSHPort
(minikube) Calling .GetSSHKeyPath
????  Userspace file server: (minikube) Calling .GetSSHUsername
ufs starting
????  Userspace file server is shutdown

????  mount failed: mount: /mnt/vda1/data/tf: mount(2) system call failed: Connection refused.
: Process exited with status 32

????  Sorry that minikube crashed. If this was unexpected, we would love to hear from you:
????  https://github.com/kubernetes/minikube/issues/new

预期挂载映射 from local:host 但是连接被拒绝。可能是一些防火墙或代理问题？

【问题讨论】：

标签： macos kubernetes minikube

【解决方案1】：

简而言之，可能的解决方案是将 minikube 更新到最新版本。您可能还需要使用 minikube delete 命令删除 minikube VM，并使用 brew upgrade && brew cask upgrade 更新其他软件，如 docker

update2: 连接问题的另一个原因是您使用了错误的 ufs 服务器 IP，通常是特定虚拟子网中的主机 IP 地址。对于不同的虚拟环境，它通常是不同的。对于--vm-driver=hyperkit，它是192.168.64.1，对于--vm-driver=virtualbox，它是192.168.99.1，因为我已经更改了virtualbox 网络默认IP 子网（192.168.56.0/24）。它也应该可以在根本不指定 --ip 参数的情况下正常工作。请看下面的详细信息。

答案的其余部分包含我使用不同 vm-drivers 和 minikube 参数进行的实验的输出。

因此，在 OSX v10.14.3 和 minikube v0.35.0 上将文件夹共享到 minikube VM 效果很好。我已经针对--vm-driver=virtualbox 和--vm-driver=hyperkit 对其进行了测试。

更新：我还使用最新版本的 minikube (v1.2.0) 对其进行了测试，两个选项为 vm-driver：hyperkit 和 @987654333 @，它的工作方式与旧 minikube 的工作方式完全相同，如下所述。但日志消息略有不同。

结果如下：

您必须安装 virtualbox 才能使用 virtualbox vm-driver 运行 minikube：

$ brew cask install virtualbox
$ brew cask install virtualbox-extension-pack

--vm-driver=virtualbox :

$ minikube start   # --vm-driver=virtualbox is default

不指定 --ip 的共享文件夹：

$ minikube mount --v=7 /Users/some_username/temp/minikube/mkshared/:/mnt/mkshared

?  Mounting /Users/some_username/temp/minikube/mkshared/ into /mnt/mkshared on the minikube VM
?  This daemon process needs to stay alive for the mount to be accessible ...
ufs starting
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 127.0.0.1 62225 <nil> <nil>}
About to run SSH command:
sudo umount /mnt/mkshared;
SSH cmd err, output: <nil>: 
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 127.0.0.1 62225 <nil> <nil>}
About to run SSH command:

sudo mkdir -p /mnt/mkshared || true;
sudo mount -t 9p -o trans=tcp,port=62422,dfltuid=1001,dfltgid=1001,version=9p2000.L,msize=262144 192.168.99.1 /mnt/mkshared;
sudo chmod 775 /mnt/mkshared || true;
2019/07/25 17:34:04 connected
2019/07/25 17:34:04 >>> 192.168.99.101:45904 Tversion tag 65535 msize 65536 version '9P2000.L'
2019/07/25 17:34:04 <<< 192.168.99.101:45904 Rversion tag 65535 msize 65536 version '9P2000'
2019/07/25 17:34:04 >>> 192.168.99.101:45904 Tattach tag 1 fid 0 afid 4294967295 uname 'nobody' nuname 0 aname ''
2019/07/25 17:34:04 <<< 192.168.99.101:45904 Rattach tag 1 aqid (759258 29c45c20 'd')
2019/07/25 17:34:04 >>> 192.168.99.101:45904 Tstat tag 1 fid 0
2019/07/25 17:34:04 <<< 192.168.99.101:45904 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 29c45c20 'd') m d775 at 0 mt 1564068830 l 96 t 0 d 0 ext )
2019/07/25 17:34:04 >>> 192.168.99.101:45904 Tstat tag 1 fid 0
2019/07/25 17:34:04 <<< 192.168.99.101:45904 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 29c45c20 'd') m d775 at 0 mt 1564068830 l 96 t 0 d 0 ext )
2019/07/25 17:34:04 >>> 192.168.99.101:45904 Twstat tag 1 fid 0 st ('' '' '' '' q (ffffffffffffffff ffffffff 'daAltL') m d775 at 4294967295 mt 4294967295 l 18446744073709551615 t 65535 d 4294967295 ext )
2019/07/25 17:34:04 <<< 192.168.99.101:45904 Rwstat tag 1

我在单独的终端窗口中运行这些命令，因为在我使用它时需要保持 ufs 服务器运行。

host-vm$ minikube ssh
minikube-vm# mount 
...<some output skipped>
192.168.99.1 on /mnt/mkshared type 9p (rw,relatime,sync,dirsync,dfltuid=1001,dfltgid=1001,access=any,msize=65536,trans=tcp,noextend,port=62365)

请注意，自动检测 ufs 服务器 IP 地址。我在下一个实验中使用它作为命令行参数--ip 的值。

指定 --ip 的共享文件夹：

$ minikube mount --ip 192.168.99.1 --v=7 /Users/some_username/temp/minikube/mkshared/:/mnt/mkshared

?  Mounting /Users/some_username/temp/minikube/mkshared/ into /mnt/mkshared on the minikube VM
?  This daemon process needs to stay alive for the mount to be accessible ...
ufs starting
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 127.0.0.1 62225 <nil> <nil>}
About to run SSH command:
sudo umount /mnt/mkshared;
SSH cmd err, output: <nil>: 
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 127.0.0.1 62225 <nil> <nil>}
About to run SSH command:

sudo mkdir -p /mnt/mkshared || true;
sudo mount -t 9p -o trans=tcp,port=62410,dfltuid=1001,dfltgid=1001,version=9p2000.L,msize=262144 192.168.99.1 /mnt/mkshared;
sudo chmod 775 /mnt/mkshared || true;
2019/07/25 17:31:37 connected
2019/07/25 17:31:37 >>> 192.168.99.101:41646 Tversion tag 65535 msize 65536 version '9P2000.L'
2019/07/25 17:31:37 <<< 192.168.99.101:41646 Rversion tag 65535 msize 65536 version '9P2000'
2019/07/25 17:31:37 >>> 192.168.99.101:41646 Tattach tag 1 fid 0 afid 4294967295 uname 'nobody' nuname 0 aname ''
2019/07/25 17:31:37 <<< 192.168.99.101:41646 Rattach tag 1 aqid (759258 29b8e141 'd')
2019/07/25 17:31:37 >>> 192.168.99.101:41646 Tstat tag 1 fid 0
2019/07/25 17:31:37 <<< 192.168.99.101:41646 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 29b8e141 'd') m d775 at 0 mt 1564068077 l 96 t 0 d 0 ext )
2019/07/25 17:31:37 >>> 192.168.99.101:41646 Tstat tag 1 fid 0
2019/07/25 17:31:37 <<< 192.168.99.101:41646 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 29b8e141 'd') m d775 at 0 mt 1564068077 l 96 t 0 d 0 ext )
2019/07/25 17:31:37 >>> 192.168.99.101:41646 Twstat tag 1 fid 0 st ('' '' '' '' q (ffffffffffffffff ffffffff 'daAltL') m d775 at 4294967295 mt 4294967295 l 18446744073709551615 t 65535 d 4294967295 ext )
2019/07/25 17:31:37 <<< 192.168.99.101:41646 Rwstat tag 1

同样，我在单独的终端窗口中运行这些命令，因为在我使用 ufs 服务器时需要它保持运行。

host-vm$ minikube ssh
minikube-vm# mount 
...<some output skipped>
192.168.99.1 on /mnt/mkshared type 9p (rw,relatime,sync,dirsync,dfltuid=1001,dfltgid=1001,access=any,msize=65536,trans=tcp,noextend,port=62365)

在共享目录中创建和编辑文件可以正常工作，在 minikube VM 中创建文件后，文件同时在主机目录中可用。它也以相反的方式运作良好。需要安装 docker-machine-driver-hyperkit

$ brew install docker-machine-driver-hyperkit

--vm-driver=hyperkit:

$ minikube start --vm-driver=hyperkit

?  minikube v0.35.0 on darwin (amd64)
?  Creating hyperkit VM (CPUs=2, Memory=2048MB, Disk=20000MB) ...
?  "minikube" IP address is 192.168.64.2
?  Configuring Docker as the container runtime ...
✨  Preparing Kubernetes environment ...
?  Pulling images required by Kubernetes v1.13.4 ...
?  Launching Kubernetes v1.13.4 using kubeadm ... 
⌛  Waiting for pods: apiserver proxy etcd scheduler controller addon-manager dns
?  Configuring cluster permissions ...
?  Verifying component health .....
?  kubectl is now configured to use "minikube"
?  Done! Thank you for using minikube!

不指定 --ip 的共享文件夹：

$ minikube mount --v=9 /Users/some_username/temp/minikube/mkshared/:/mnt/mkshared

Found binary path at /usr/local/bin/docker-machine-driver-hyperkit
Launching plugin server for driver hyperkit
Plugin server listening at address 127.0.0.1:63027
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .DriverName
?  Mounting /Users/some_username/temp/minikube/mkshared/ into /mnt/mkshared on the minikube VM
?  This daemon process needs to stay alive for the mount to be accessible ...
Found binary path at /usr/local/bin/docker-machine-driver-hyperkit
Launching plugin server for driver hyperkit
ufs starting
Plugin server listening at address 127.0.0.1:63034
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetSSHHostname
(minikube) Calling .GetSSHPort
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHUsername
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 192.168.64.2 22 <nil> <nil>}
About to run SSH command:
sudo umount /mnt/mkshared;
SSH cmd err, output: Process exited with status 32: umount: /mnt/mkshared: no mount point specified.

(minikube) Calling .GetSSHHostname
(minikube) Calling .GetSSHPort
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHUsername
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 192.168.64.2 22 <nil> <nil>}
About to run SSH command:

sudo mkdir -p /mnt/mkshared || true;
sudo mount -t 9p -o trans=tcp,port=63031,dfltuid=1001,dfltgid=1001,version=9p2000.L,msize=262144 192.168.64.1 /mnt/mkshared;
sudo chmod 775 /mnt/mkshared || true;
2019/07/26 10:51:15 connected
2019/07/26 10:51:15 >>> 192.168.64.2:47342 Tversion tag 65535 msize 65536 version '9P2000.L'
2019/07/26 10:51:15 <<< 192.168.64.2:47342 Rversion tag 65535 msize 65536 version '9P2000'
2019/07/26 10:51:15 >>> 192.168.64.2:47342 Tattach tag 1 fid 0 afid 4294967295 uname 'nobody' nuname 0 aname ''
2019/07/26 10:51:15 <<< 192.168.64.2:47342 Rattach tag 1 aqid (759258 29c66350 'd')
2019/07/26 10:51:15 >>> 192.168.64.2:47342 Tstat tag 1 fid 0
2019/07/26 10:51:15 <<< 192.168.64.2:47342 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 29c66350 'd') m d775 at 0 mt 1564068963 l 96 t 0 d 0 ext )
2019/07/26 10:51:15 >>> 192.168.64.2:47342 Tstat tag 1 fid 0
2019/07/26 10:51:15 <<< 192.168.64.2:47342 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 29c66350 'd') m d775 at 0 mt 1564068963 l 96 t 0 d 0 ext )
2019/07/26 10:51:15 >>> 192.168.64.2:47342 Twstat tag 1 fid 0 st ('' '' '' '' q (ffffffffffffffff ffffffff 'daAltL') m d775 at 4294967295 mt 4294967295 l 18446744073709551615 t 65535 d 4294967295 ext )
2019/07/26 10:51:15 <<< 192.168.64.2:47342 Rwstat tag 1
SSH cmd err, output: <nil>:

单独的终端窗口：

host-vm$ minikube ssh
minikube-vm# mount 
...<some output skipped>
192.168.64.1 on /mnt/mkshared type 9p (rw,relatime,sync,dirsync,dfltuid=1001,dfltgid=1001,access=any,msize=65536,trans=tcp,noextend,port=63031)

请注意，ufs 服务器 IP 地址是自动检测的，并且与 hyperkit vm-driver 不同。这一次，我还使用下一个实验中挂载输出中的相同 IP 地址作为命令行参数 --ip 的值。

指定 --ip 的共享文件夹：

$ minikube mount --ip=192.168.64.1 --v=9 /Users/some_username/temp/minikube/mkshared/:/mnt/mkshared

Found binary path at /usr/local/bin/docker-machine-driver-hyperkit
Launching plugin server for driver hyperkit
Plugin server listening at address 127.0.0.1:63168
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .DriverName
?  Mounting /Users/some_username/temp/minikube/mkshared/ into /mnt/mkshared on the minikube VM
?  This daemon process needs to stay alive for the mount to be accessible ...
ufs starting
Found binary path at /usr/local/bin/docker-machine-driver-hyperkit
Launching plugin server for driver hyperkit
Plugin server listening at address 127.0.0.1:63175
() Calling .GetVersion
Using API Version  1
() Calling .SetConfigRaw
() Calling .GetMachineName
(minikube) Calling .GetSSHHostname
(minikube) Calling .GetSSHPort
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHUsername
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 192.168.64.2 22 <nil> <nil>}
About to run SSH command:
sudo umount /mnt/mkshared;
SSH cmd err, output: <nil>: 
(minikube) Calling .GetSSHHostname
(minikube) Calling .GetSSHPort
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHKeyPath
(minikube) Calling .GetSSHUsername
Using SSH client type: native
&{{{<nil> 0 [] [] []} docker [0x143d0e0] 0x143d0b0  [] 0s} 192.168.64.2 22 <nil> <nil>}
About to run SSH command:

sudo mkdir -p /mnt/mkshared || true;
sudo mount -t 9p -o trans=tcp,port=63172,dfltuid=1001,dfltgid=1001,version=9p2000.L,msize=262144 192.168.64.1 /mnt/mkshared;
sudo chmod 775 /mnt/mkshared || true;
2019/07/26 10:59:45 connected
2019/07/26 10:59:45 >>> 192.168.64.2:34050 Tversion tag 65535 msize 65536 version '9P2000.L'
2019/07/26 10:59:45 <<< 192.168.64.2:34050 Rversion tag 65535 msize 65536 version '9P2000'
2019/07/26 10:59:45 >>> 192.168.64.2:34050 Tattach tag 1 fid 0 afid 4294967295 uname 'nobody' nuname 0 aname ''
2019/07/26 10:59:45 <<< 192.168.64.2:34050 Rattach tag 1 aqid (759258 2d80dc96 'd')
2019/07/26 10:59:45 >>> 192.168.64.2:34050 Tstat tag 1 fid 0
2019/07/26 10:59:45 <<< 192.168.64.2:34050 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 2d80dc96 'd') m d775 at 0 mt 1564131515 l 96 t 0 d 0 ext )
2019/07/26 10:59:45 >>> 192.168.64.2:34050 Tstat tag 1 fid 0
2019/07/26 10:59:45 <<< 192.168.64.2:34050 Rstat tag 1 st ('mkshared' 'some_username' '1720083983' '' q (759258 2d80dc96 'd') m d775 at 0 mt 1564131515 l 96 t 0 d 0 ext )
2019/07/26 10:59:45 >>> 192.168.64.2:34050 Twstat tag 1 fid 0 st ('' '' '' '' q (ffffffffffffffff ffffffff 'daAltL') m d775 at 4294967295 mt 4294967295 l 18446744073709551615 t 65535 d 4294967295 ext )
2019/07/26 10:59:45 <<< 192.168.64.2:34050 Rwstat tag 1
SSH cmd err, output: <nil>: 
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Twalk tag 1 fid 0 newfid 1 0:'somefile_from_minikube.txt' 
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rwalk tag 1 (75b56b 2d80dc2b '') 
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Tstat tag 1 fid 1
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rstat tag 1 st ('somefile_from_minikube.txt' 'some_username' '1720083983' '' q (75b56b 2d80dc2b '') m 644 at 0 mt 1564131515 l 8140 t 0 d 0 ext )
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Twalk tag 1 fid 1 newfid 2 
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rwalk tag 1 
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Topen tag 1 fid 2 mode 0
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Ropen tag 1 qid (75b56b 2d80dc2b '') iounit 0
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Tstat tag 1 fid 1
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rstat tag 1 st ('somefile_from_minikube.txt' 'some_username' '1720083983' '' q (75b56b 2d80dc2b '') m 644 at 0 mt 1564131515 l 8140 t 0 d 0 ext )
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Tread tag 1 fid 2 offset 0 count 65512
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rread tag 1 count 8140
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Tread tag 1 fid 2 offset 8140 count 65512
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rread tag 1 count 0
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Tread tag 1 fid 2 offset 8140 count 65512
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rread tag 1 count 0
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Tclunk tag 1 fid 2
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rclunk tag 1
2019/07/26 10:59:53 >>> 192.168.64.2:34050 Tclunk tag 1 fid 1
2019/07/26 10:59:53 <<< 192.168.64.2:34050 Rclunk tag 1

单独的终端窗口：

$ minikube ssh
#mount 
...<some output skipped>
192.168.64.1 on /mnt/mkshared type 9p (rw,relatime,sync,dirsync,dfltuid=1001,dfltgid=1001,access=any,msize=65536,trans=tcp,noextend,port=63172)

访问共享文件夹中的现有内容以及创建新文件效果很好，文件在 minikube VM 中创建后同时在主机目录中可用。

【讨论】：

【解决方案2】：

我将把它留在这里，因为我在 Fedora 32 上遇到过类似的问题：

mount failed: mount with cmd /bin/bash -c "sudo mount -t 9p -o dfltgid=$(grep ^docker: /etc/group | cut -d: -f3),dfltuid=$(id -u docker),msize=262144,port=38811,trans=tcp,version=9p2000.L 192.168.39.1 /data/pvtheme" : /bin/bash -c "sudo mount -t 9p -o dfltgid=$(grep ^docker: /etc/group | cut -d: -f3),dfltuid=$(id -u docker),msize=262144,port=38811,trans=tcp,version=9p2000.L 192.168.39.1 /data/pvtheme": Process exited with status 32
stdout:

stderr:
mount: /data/pvtheme: mount(2) system call failed: Connection refused.

感谢这条评论，我解决了：https://github.com/kubernetes/minikube/issues/4726#issuecomment-510217223

这是一个firewalld 问题：

$ sudo firewall-cmd --permanent --zone=libvirt --add-rich-rule='rule family="ipv4" source address="192.168.39.0/24" accept'

$ sudo firewall-cmd --reload

更改后，这是我的firewalld 配置：

firewall-cmd --zone=libvirt --list-all
libvirt (active)
  target: ACCEPT
  icmp-block-inversion: no
  interfaces: crc virbr0 virbr1
  sources: 
  services: dhcp dhcpv6 dns ssh tftp
  ports: 
  protocols: icmp ipv6-icmp
  masquerade: no
  forward-ports: 
  source-ports: 
  icmp-blocks: 
  rich rules: 
    rule family="ipv4" source address="192.168.39.0/24" accept
    rule priority="32767" reject

【讨论】：