Helm 图表无限期停留在 PodInitializing 状态

Question

我在家里的 Ubuntu 服务器上运行 microk8s 集群，并将它连接到本地 NAS 服务器以进行持久存储。我一直把它作为我个人学习 Kubernetes 的试验场，但我似乎在每一步都遇到了一个又一个的问题。

我已经安装了NFS Client Provisioner Helm 图表，我已经确认它可以工作 - 它会在我的 NAS 服务器上动态配置 PVC。我后来能够成功安装Postgres Helm 图表，或者我是这么认为的。创建它后，我能够使用 SQL 客户端连接到它，我感觉很好。

直到几天后，我才注意到 pod 显示 0/1 容器已准备就绪。尽管有趣的是，nfs-client-provisioner pod 仍然显示 1/1。长话短说：我已经删除/清除了 Postgres Helm 图表，并尝试重新安装它，但现在它不再工作了。事实上，我尝试部署的任何新东西都不起作用。一切看起来都可以正常工作，但随后就永远挂在 Init 或 ContainerCreating 上。

特别是使用 Postgres，我一直在运行的命令是这样的：

helm install --name postgres stable/postgresql -f postgres.yaml

我的postgres.yaml 文件如下所示：

persistence:
    storageClass: nfs-client
    accessMode: ReadWriteMany
    size: 2Gi

但如果我执行kubectl get pods，我仍然可以看到：

NAME                    READY  STATUS    RESTARTS  AGE
nfs-client-provisioner  1/1    Running   1         11d
postgres-postgresql-0   0/1    Init:0/1  0         3h51m

如果我执行kubectl describe pod postgres-postgresql-0，这是输出：

Name:               postgres-postgresql-0
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               stjohn/192.168.1.217
Start Time:         Thu, 28 Mar 2019 12:51:02 -0500
Labels:             app=postgresql
                chart=postgresql-3.11.7
                controller-revision-hash=postgres-postgresql-5bfb9cc56d
                heritage=Tiller
                release=postgres
                role=master
                statefulset.kubernetes.io/pod-name=postgres-postgresql-0
Annotations:        <none>
Status:             Pending
IP:                 
Controlled By:      StatefulSet/postgres-postgresql
Init Containers:
  init-chmod-data:
    Container ID:  
    Image:         docker.io/bitnami/minideb:latest
    Image ID:      
    Port:          <none>
    Host Port:     <none>
    Command:
      sh
      -c
      chown -R 1001:1001 /bitnami
      if [ -d /bitnami/postgresql/data ]; then
    chmod  0700 /bitnami/postgresql/data;
      fi

    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Requests:
      cpu:        250m
      memory:     256Mi
    Environment:  <none>
    Mounts:
      /bitnami/postgresql from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-h4gph (ro)
Containers:
  postgres-postgresql:
    Container ID:   
    Image:          docker.io/bitnami/postgresql:10.7.0
    Image ID:       
    Port:           5432/TCP
    Host Port:      0/TCP
    State:          Waiting
      Reason:       PodInitializing
    Ready:          False
    Restart Count:  0
    Requests:
      cpu:      250m
      memory:   256Mi
    Liveness:   exec [sh -c exec pg_isready -U "postgres" -h localhost] delay=30s timeout=5s period=10s #success=1 #failure=6
    Readiness:  exec [sh -c exec pg_isready -U "postgres" -h localhost] delay=5s timeout=5s period=10s #success=1 #failure=6
    Environment:
      PGDATA:             /bitnami/postgresql
      POSTGRES_USER:      postgres
      POSTGRES_PASSWORD:  <set to the key 'postgresql-password' in secret 'postgres-postgresql'>  Optional: false
    Mounts:
      /bitnami/postgresql from data (rw)
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-h4gph (ro)
Conditions:
  Type              Status
  Initialized       False 
  Ready             False 
  ContainersReady   False 
  PodScheduled      True 
Volumes:
  data:
    Type:       PersistentVolumeClaim (a reference to a PersistentVolumeClaim in the same namespace)
    ClaimName:  data-postgres-postgresql-0
    ReadOnly:   false
  default-token-h4gph:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-h4gph
    Optional:    false
QoS Class:       Burstable
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
             node.kubernetes.io/unreachable:NoExecute for 300s
Events:          <none>

如果我执行kubectl get pod postgres-postgresql-0 -o yaml，输出如下：

apiVersion: v1
kind: Pod
metadata:
  creationTimestamp: "2019-03-28T17:51:02Z"
  generateName: postgres-postgresql-
  labels:
    app: postgresql
    chart: postgresql-3.11.7
    controller-revision-hash: postgres-postgresql-5bfb9cc56d
    heritage: Tiller
    release: postgres
    role: master
    statefulset.kubernetes.io/pod-name: postgres-postgresql-0
  name: postgres-postgresql-0
  namespace: default
  ownerReferences:
  - apiVersion: apps/v1
    blockOwnerDeletion: true
    controller: true
    kind: StatefulSet
    name: postgres-postgresql
    uid: 0d3ef673-5182-11e9-bf14-b8975a0ca30c
  resourceVersion: "1953329"
  selfLink: /api/v1/namespaces/default/pods/postgres-postgresql-0
  uid: 0d4dfb56-5182-11e9-bf14-b8975a0ca30c
spec:
  containers:
  - env:
    - name: PGDATA
      value: /bitnami/postgresql
    - name: POSTGRES_USER
      value: postgres
    - name: POSTGRES_PASSWORD
      valueFrom:
    secretKeyRef:
      key: postgresql-password
      name: postgres-postgresql
    image: docker.io/bitnami/postgresql:10.7.0
    imagePullPolicy: Always
    livenessProbe:
      exec:
    command:
    - sh
    - -c
    - exec pg_isready -U "postgres" -h localhost
      failureThreshold: 6
      initialDelaySeconds: 30
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 5
    name: postgres-postgresql
    ports:
    - containerPort: 5432
      name: postgresql
      protocol: TCP
    readinessProbe:
      exec:
    command:
    - sh
    - -c
    - exec pg_isready -U "postgres" -h localhost
      failureThreshold: 6
      initialDelaySeconds: 5
      periodSeconds: 10
      successThreshold: 1
      timeoutSeconds: 5
    resources:
      requests:
    cpu: 250m
    memory: 256Mi
    securityContext:
      procMount: Default
      runAsUser: 1001
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /bitnami/postgresql
      name: data
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-h4gph
      readOnly: true
  dnsPolicy: ClusterFirst
  enableServiceLinks: true
  hostname: postgres-postgresql-0
  initContainers:
  - command:
    - sh
    - -c
    - |
      chown -R 1001:1001 /bitnami
      if [ -d /bitnami/postgresql/data ]; then
    chmod  0700 /bitnami/postgresql/data;
      fi
    image: docker.io/bitnami/minideb:latest
    imagePullPolicy: Always
    name: init-chmod-data
    resources:
      requests:
    cpu: 250m
    memory: 256Mi
    securityContext:
      procMount: Default
      runAsUser: 0
    terminationMessagePath: /dev/termination-log
    terminationMessagePolicy: File
    volumeMounts:
    - mountPath: /bitnami/postgresql
      name: data
    - mountPath: /var/run/secrets/kubernetes.io/serviceaccount
      name: default-token-h4gph
      readOnly: true
  nodeName: stjohn
  priority: 0
  restartPolicy: Always
  schedulerName: default-scheduler
  securityContext:
    fsGroup: 1001
  serviceAccount: default
  serviceAccountName: default
  subdomain: postgres-postgresql-headless
  terminationGracePeriodSeconds: 30
  tolerations:
  - effect: NoExecute
    key: node.kubernetes.io/not-ready
    operator: Exists
    tolerationSeconds: 300
  - effect: NoExecute
    key: node.kubernetes.io/unreachable
    operator: Exists
    tolerationSeconds: 300
  volumes:
  - name: data
    persistentVolumeClaim:
      claimName: data-postgres-postgresql-0
  - name: default-token-h4gph
    secret:
      defaultMode: 420
      secretName: default-token-h4gph
status:
  conditions:
  - lastProbeTime: null
    lastTransitionTime: "2019-03-28T17:51:02Z"
    message: 'containers with incomplete status: [init-chmod-data]'
    reason: ContainersNotInitialized
    status: "False"
    type: Initialized
  - lastProbeTime: null
    lastTransitionTime: "2019-03-28T17:51:02Z"
    message: 'containers with unready status: [postgres-postgresql]'
    reason: ContainersNotReady
    status: "False"
    type: Ready
  - lastProbeTime: null
    lastTransitionTime: "2019-03-28T17:51:02Z"
    message: 'containers with unready status: [postgres-postgresql]'
    reason: ContainersNotReady
    status: "False"
    type: ContainersReady
  - lastProbeTime: null
    lastTransitionTime: "2019-03-28T17:51:02Z"
    status: "True"
    type: PodScheduled
  containerStatuses:
  - image: docker.io/bitnami/postgresql:10.7.0
    imageID: ""
    lastState: {}
    name: postgres-postgresql
    ready: false
    restartCount: 0
    state:
      waiting:
    reason: PodInitializing
  hostIP: 192.168.1.217
  initContainerStatuses:
  - image: docker.io/bitnami/minideb:latest
    imageID: ""
    lastState: {}
    name: init-chmod-data
    ready: false
    restartCount: 0
    state:
      waiting:
    reason: PodInitializing
  phase: Pending
  qosClass: Burstable
  startTime: "2019-03-28T17:51:02Z"

我没有看到任何明显的东西能够确定可能发生的事情。而且我已经重新启动了服务器，看看是否有帮助。有什么想法吗？为什么我的容器无法启动？

Answer 1

您的 initContainer 似乎卡在 PodInitializing 状态。最可能的情况是您的 PVC 尚未准备好。我建议您 describe 您的 data-postgres-postgresql-0 PVC 以确保该卷已实际配置并处于 READY 状态。您的 NFS 配置程序可能正在工作，但由于错误可能尚未创建特定的 PV/PVC。我在使用 AWS 的 EFS 预配器时遇到了类似的现象。

Answer 2

您可以使用 kubectl 的 event 命令。这将为您的 pod 提供事件。

要过滤特定的 pod，您可以使用字段选择器：

kubectl get event --namespace abc-namespace --field-selector involvedObject.name=my-pod-zl6m6