kubectl 创建秘密通用 InternalError PermissionDenied

Question

我已经配置了对我的 K8s 集群的访问，设置了所有需要的 pod 和服务，使用 YAML 文件创建了秘密，但是这个简单的命令：

kubectl create secret generic my-secret --from-literal=key1=supersecret --from-literal=key2=topsecret
kubectl create secret generic my-secret --from-file=path/to/bar

导致错误：

来自服务器的错误 (InternalError)：发生内部错误：rpc 错误：code = Internal desc = kms 服务加密错误：rpc 错误：code = PermissionDenied desc = Permission denied

我该如何解决这个问题？？

更多细节：集群在Yandex cloud 上运行。当然，我已经写信给支持，但我希望在这里得到更快的解决方案回复。

更新。。一些角色信息：

kubectl get rolebindings,clusterrolebindings --all-namespaces


NAMESPACE    NAME    ROLE    AGE
kube-public   rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer    Role/system:controller:bootstrap-signer    15d
kube-system   rolebinding.rbac.authorization.k8s.io/cluster-autoscaler    Role/cluster-autoscaler    15d
kube-system   rolebinding.rbac.authorization.k8s.io/metrics-server-auth-reader    Role/extension-apiserver-authentication-reader    15d
kube-system   rolebinding.rbac.authorization.k8s.io/node-metrics-agent-rb    Role/node-metrics-agent    15d
kube-system   rolebinding.rbac.authorization.k8s.io/system::extension-apiserver-authentication-reader   Role/extension-apiserver-authentication-reader    15d
kube-system   rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-controller-manager    Role/system::leader-locking-kube-controller-manager   15d
kube-system   rolebinding.rbac.authorization.k8s.io/system::leader-locking-kube-scheduler    Role/system::leader-locking-kube-scheduler    15d
kube-system   rolebinding.rbac.authorization.k8s.io/system:controller:bootstrap-signer    Role/system:controller:bootstrap-signer    15d
kube-system   rolebinding.rbac.authorization.k8s.io/system:controller:cloud-provider    Role/system:controller:cloud-provider    15d
kube-system   rolebinding.rbac.authorization.k8s.io/system:controller:token-cleaner    Role/system:controller:token-cleaner    15d
monitoring    rolebinding.rbac.authorization.k8s.io/loki    Role/loki    14d
monitoring    rolebinding.rbac.authorization.k8s.io/loki-promtail    Role/loki-promtail    14d
monitoring    rolebinding.rbac.authorization.k8s.io/prom-grafana    Role/prom-grafana    14d
monitoring    rolebinding.rbac.authorization.k8s.io/prom-grafana-test    Role/prom-grafana-test    14d
monitoring    rolebinding.rbac.authorization.k8s.io/prom-kube-prometheus-stack-alertmanager    Role/prom-kube-prometheus-stack-alertmanager    14d


NAMESPACE   NAME    ROLE    AGE
    clusterrolebinding.rbac.authorization.k8s.io/ccm-binding    ClusterRole/cluster-admin    15d
    clusterrolebinding.rbac.authorization.k8s.io/cluster-admin    ClusterRole/cluster-admin    15d
    clusterrolebinding.rbac.authorization.k8s.io/cluster-autoscaler    ClusterRole/cluster-autoscaler    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-attacher-binding    ClusterRole/external-attacher-role    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-csinodeinfos-reader-binding    ClusterRole/csinodeinfos-reader    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-csinodes-reader-binding    ClusterRole/csinodes-reader    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-driver-registrar-binding    ClusterRole/cluster-driver-registrar-role    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-endpoints-reader-binding    ClusterRole/endpoints-operator    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-leases-operator-binding    ClusterRole/leases-operator    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-provisioner-binding    ClusterRole/external-provisioner-role    15d
    clusterrolebinding.rbac.authorization.k8s.io/csi-snapshotter-binding    ClusterRole/external-snapshotter-role    15d
    clusterrolebinding.rbac.authorization.k8s.io/event-logger-rb    ClusterRole/view    15d
    clusterrolebinding.rbac.authorization.k8s.io/loki-promtail-clusterrolebinding    ClusterRole/loki-promtail-clusterrole    14d
    clusterrolebinding.rbac.authorization.k8s.io/metrics-server:system:auth-delegator    ClusterRole/system:auth-delegator    15d
    clusterrolebinding.rbac.authorization.k8s.io/nginx-ingress-nginx-ingress    ClusterRole/nginx-ingress-nginx-ingress    14d
    clusterrolebinding.rbac.authorization.k8s.io/node-metrics-agent-rb    ClusterRole/node-metrics-agent    15d
    clusterrolebinding.rbac.authorization.k8s.io/npd-binding    ClusterRole/system:node-problem-detector    15d
    clusterrolebinding.rbac.authorization.k8s.io/npd-ds-binding    ClusterRole/system:node-problem-detector    15d
    clusterrolebinding.rbac.authorization.k8s.io/prom-grafana-clusterrolebinding    ClusterRole/prom-grafana-clusterrole    14d
    clusterrolebinding.rbac.authorization.k8s.io/prom-kube-prometheus-stack-operator    ClusterRole/prom-kube-prometheus-stack-operator    14d
    clusterrolebinding.rbac.authorization.k8s.io/prom-kube-prometheus-stack-operator-psp    ClusterRole/prom-kube-prometheus-stack-operator-psp    14d
    clusterrolebinding.rbac.authorization.k8s.io/prom-kube-prometheus-stack-prometheus    ClusterRole/prom-kube-prometheus-stack-prometheus    14d
    clusterrolebinding.rbac.authorization.k8s.io/prom-kube-prometheus-stack-prometheus-psp    ClusterRole/prom-kube-prometheus-stack-prometheus-psp    14d
    clusterrolebinding.rbac.authorization.k8s.io/prom-kube-state-metrics    ClusterRole/prom-kube-state-metrics    14d
    clusterrolebinding.rbac.authorization.k8s.io/psp-prom-kube-state-metrics    ClusterRole/psp-prom-kube-state-metrics    14d
    clusterrolebinding.rbac.authorization.k8s.io/psp-prom-prometheus-node-exporter    ClusterRole/psp-prom-prometheus-node-exporter    14d
    clusterrolebinding.rbac.authorization.k8s.io/system:basic-user    ClusterRole/system:basic-user    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:attachdetach-controller    ClusterRole/system:controller:attachdetach-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:certificate-controller    ClusterRole/system:controller:certificate-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:clusterrole-aggregation-controller   ClusterRole/system:controller:clusterrole-aggregation-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:cronjob-controller    ClusterRole/system:controller:cronjob-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:daemon-set-controller    ClusterRole/system:controller:daemon-set-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:deployment-controller    ClusterRole/system:controller:deployment-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:disruption-controller    ClusterRole/system:controller:disruption-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:endpoint-controller    ClusterRole/system:controller:endpoint-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:endpointslice-controller    ClusterRole/system:controller:endpointslice-controller    14d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:expand-controller    ClusterRole/system:controller:expand-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:generic-garbage-collector    ClusterRole/system:controller:generic-garbage-collector    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:horizontal-pod-autoscaler    ClusterRole/system:controller:horizontal-pod-autoscaler    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:job-controller    ClusterRole/system:controller:job-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:namespace-controller    ClusterRole/system:controller:namespace-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:node-controller    ClusterRole/system:controller:node-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:persistent-volume-binder    ClusterRole/system:controller:persistent-volume-binder    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:pod-garbage-collector    ClusterRole/system:controller:pod-garbage-collector    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:pv-protection-controller    ClusterRole/system:controller:pv-protection-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:pvc-protection-controller    ClusterRole/system:controller:pvc-protection-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:replicaset-controller    ClusterRole/system:controller:replicaset-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:replication-controller    ClusterRole/system:controller:replication-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:resourcequota-controller    ClusterRole/system:controller:resourcequota-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:route-controller    ClusterRole/system:controller:route-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-account-controller    ClusterRole/system:controller:service-account-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:service-controller    ClusterRole/system:controller:service-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:statefulset-controller    ClusterRole/system:controller:statefulset-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:controller:ttl-controller    ClusterRole/system:controller:ttl-controller    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:coredns    ClusterRole/system:coredns    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:discovery    ClusterRole/system:discovery    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:kube-controller-manager    ClusterRole/system:kube-controller-manager    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:kube-dns    ClusterRole/system:kube-dns    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:kube-dns-autoscaler    ClusterRole/system:kube-dns-autoscaler    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:kube-proxy    ClusterRole/system:node-proxier    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:kube-scheduler    ClusterRole/system:kube-scheduler    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:metrics-server    ClusterRole/system:metrics-server    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:node    ClusterRole/system:node    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:node-proxier    ClusterRole/system:node-proxier    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:public-info-viewer    ClusterRole/system:public-info-viewer    15d
    clusterrolebinding.rbac.authorization.k8s.io/system:volume-scheduler    ClusterRole/system:volume-scheduler    15d
    clusterrolebinding.rbac.authorization.k8s.io/yc:admin    ClusterRole/cluster-admin    15d
    clusterrolebinding.rbac.authorization.k8s.io/yc:alpha:auto-approve-csrs-for-group    ClusterRole/system:certificates.k8s.io:certificatesigningrequests:nodeclient    15d
    clusterrolebinding.rbac.authorization.k8s.io/yc:alpha:auto-approve-renewals-for-nodes    ClusterRole/system:certificates.k8s.io:certificatesigningrequests:selfnodeclient   15d
    clusterrolebinding.rbac.authorization.k8s.io/yc:alpha:create-csrs-for-bootstrapping    ClusterRole/system:node-bootstrapper    15d
    clusterrolebinding.rbac.authorization.k8s.io/yc:editor    ClusterRole/edit    15d
    clusterrolebinding.rbac.authorization.k8s.io/yc:viewer    ClusterRole/view    15d

Answer 1

我找到了解决方案：我必须在 Yandex.Cloud 项目目录的设置中将角色 kms.keys.encrypterDecrypter 设置为用于控制 Kubernetes 集群的服务帐户。