通过 microk8s 获取“x509：由未知权威签署的证书”答案

【问题标题】：Getting "x509: certificate signed by unknown authority" by microk8s通过 microk8s 获取“x509：由未知权威签署的证书”
【发布时间】：2020-11-05 20:20:27
【问题描述】：

我想将 microk8s 与 私有注册表 一起使用，但拉取映像不起作用（我使用的是自签名证书）：

root@master-1:/var/snap/microk8s/common/var/lib/containerd# microk8s.ctr --debug images pull priv.repo:5000/busybox/hellomicrok8s:latest
DEBU[0000] fetching                                      image="priv.repo:5000/busybox/hellomicrok8s:latest"
DEBU[0000] resolving                                     host="priv.repo:5000"
DEBU[0000] do request                                    host="priv.repo:5000" request.header.accept="application/vnd.docker.distribution.manifest.v2+json, application/vnd.docker.distribution.manifest.list.v2+json, application/vnd.oci.image.manifest.v1+json, application/vnd.oci.image.index.v1+json, */*" request.header.user-agent=containerd/v1.3.4 request.method=HEAD url="https://priv.repo:5000/v2/busybox/hellomicrok8s/manifests/latest"
ctr: failed to resolve reference "priv.repo:5000/busybox/hellomicrok8s:latest": failed to do request: Head "https://priv.repo:5000/v2/busybox/hellomicrok8s/manifests/latest": x509: certificate signed by unknown authority

这是我的 containerd-template.tom：

root@master-1:/var/snap/microk8s/common/var/lib/containerd# cat /var/snap/microk8s/current/args/containerd-template.toml
version = 2
oom_score = 0

[grpc]
  uid = 0
  gid = 0
  max_recv_message_size = 16777216
  max_send_message_size = 16777216

[debug]
  address = ""
  uid = 0
  gid = 0

[metrics]
  address = "127.0.0.1:1338"
  grpc_histogram = false

[cgroup]
  path = ""

[plugins."io.containerd.grpc.v1.cri"]

  stream_server_address = "127.0.0.1"
  stream_server_port = "0"
  enable_selinux = false
  sandbox_image = "k8s.gcr.io/pause:3.1"
  stats_collect_period = 10
  enable_tls_streaming = false
  max_container_log_line_size = 16384

  [plugins."io.containerd.grpc.v1.cri".containerd]
    snapshotter = "${SNAPSHOTTER}"
    no_pivot = false
    default_runtime_name = "${RUNTIME}"

    [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.runc]
      runtime_type = "io.containerd.runc.v1"

    [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia-container-runtime]
      runtime_type = "io.containerd.runc.v1"

      [plugins."io.containerd.grpc.v1.cri".containerd.runtimes.nvidia-container-runtime.options]
        BinaryName = "nvidia-container-runtime"

  [plugins."io.containerd.grpc.v1.cri".cni]
    bin_dir = "${SNAP}/opt/cni/bin"
    conf_dir = "${SNAP_DATA}/args/cni-network"

  [plugins."io.containerd.grpc.v1.cri".registry]

    [plugins."io.containerd.grpc.v1.cri".registry.mirrors]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors."docker.io"]
        endpoint = ["https://registry-1.docker.io", ]
      [plugins."io.containerd.grpc.v1.cri".registry.mirrors."priv.repo:5000"]
        endpoint = ["https://priv.repo:5000"]

我通过systemctl restart snap.microk8s.daemon-containerd.service && microk8s.stop && microk8s.start 重新启动了 microk8s。命令docker login docker https://priv.repo:5000 正在运行，我可以通过docker pull priv.repo:5000/busybox/hellomicrok8s:latest 拉取该图像。你知道它为什么不工作吗？

提前致谢！

编辑：

这也是设置的：

root@master-1:/var/snap/microk8s/common/var/lib/containerd# cat /etc/docker/daemon.json
{
    "insecure-registries" : ["priv.repo:5000"]
}

EDIT1：

这是有效的：microk8s.ctr --debug images pull -u ???:??? --skip-verify priv.repo:5000/busybox/hellomicrok8s:latest。我应该如何设置--skip-verify，因为当我通过microk8s kubectl apply -f ... 创建一个pod 时仍然得到x509: certificate signed by unknown authority。

【问题讨论】：

您将存储库的证书安装在哪里？
我正在使用 haproxy 的 ssl 终止。我应该将它们安装在某个地方吗？

标签： docker ssl kubernetes kubectl microk8s

【解决方案1】：

我将我的 crt 文件添加到 /etc/ssl/certs（在主节点上），它开始工作了。

顺便说一句，我不需要 containerd-template.tom 文件中新添加的行。

【讨论】：

【解决方案2】：

我遇到了同样的问题，下面的这些命令可能会为其他人解决这个问题

openssl s_client -showcerts -connect <IP>:<PORT>< /dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ca.crt
cp ca.crt /etc/ssl/certs
update-ca-certificates

【讨论】：

【解决方案3】：

如果你使用的是 ubuntu microk8s cert-manager，你可以像这样获取证书并安装它：

找到正确的证书名称（您可以有多个）

microk8s kubectl get secrets -n cert-manager  --field-selector type=kubernetes.io/tls

如果正确的名称是例如dev-da

microk8s kubectl -n cert-manager get secrets dev-ca -o jsonpath='{.data.ca\.crt}' | base64 -d  > cert-manager-ca.crt
sudo cp cert-manager-ca.crt /usr/local/share/ca-certificates/
sudo update-ca-certificates

此后，您可以使用 curl 检查证书是否安装正确。而当 genstart microk8s 时。

microk8s stop && microk8s start

【讨论】：