【问题标题】:Kubernetes statefulset with NFS persistent volume具有 NFS 持久卷的 Kubernetes 状态集
【发布时间】:2021-01-30 19:04:47
【问题描述】:

我有一个kubernetes 集群,并且我有一个简单的mongodb 部署和NFS 持久卷集。它工作得很好,但是由于数据库之类的资源是stateful,所以我想用Statefulset 代替mongodb,但现在的问题是,当我浏览文档时,statefulset 有volumeClaimTemplates 而不是volumes(在部署中)。

但现在问题来了。

在deployment 中这样做:

PersistentVolume -> PersistentVolumeClaim -> Deployment

但是我们如何在Statefulset 中做到这一点呢?

是这样的吗:

volumeClaimTemplates -> StatefulSet

如何为volumeClaimTemplates 设置PersistentVolume。如果我们不将PersistentVolume 用于StatefulSet,它是如何创建卷的,它在哪里创建卷?是否在 host 机器中(即 kubernetes 工作节点)?

因为我有一个单独的 NFS 预配器用于 mongodb 部署(replicasset=1),我如何使用与 StatefulSet 相同的设置?

这是我的mongo-deployment.yaml -> 我将把它转换成一个有状态集,如第二个代码 sn-p (mongo-stateful.yaml) 所示

  1. mongo-deployment.yaml
<omitted>
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: task-pv-volume
  labels:
    name: mynfs # name can be anything
spec:
  storageClassName: manual # same storage class as pvc
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteMany
  nfs:
    server: <nfs-server-ip>
    path: "/srv/nfs/mydata" 
---
apiVersion: v1
kind: PersistentVolumeClaim
metadata:
  name: task-pv-claim
spec:
  storageClassName: manual
  accessModes:
    - ReadWriteMany #  must be the same as PersistentVolume
  resources:
    requests:
      storage: 1Gi          
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-deployment    
  labels:
    name: mongodb
spec:
  selector:
    matchLabels:
      app: mongodb
  replicas: 1
  template:
    metadata:
      labels: 
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo
        ports:
        -  containerPort: 27017
        ... # omitted some parts for easy reading
        volumeMounts:
        - name: data  
          mountPath: /data/db
      volumes: 
        - name: data
          persistentVolumeClaim: 
            claimName: task-pv-claim    
  1. mongo-stateful.yaml
---
apiVersion: v1
kind: PersistentVolume
metadata:
  name: task-pv-volume
  labels:
    name: mynfs # name can be anything
spec:
  storageClassName: manual # same storage class as pvc
  capacity:
    storage: 10Gi
  accessModes:
    - ReadWriteOnce
  nfs:
    server: <nfs-server-ip>
    path: "/srv/nfs/mydata" 
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: mongodb-statefulset
spec:
  selector:
    matchLabels:
      name: mongodb-statefulset
  serviceName: mongodb-statefulset
  replicas: 2
  template:
    metadata:
      labels:
        name: mongodb-statefulset
    spec:
      terminationGracePeriodSeconds: 10
      containers:
      - name: mongodb
        image: mongo:3.6.4
        ports:
        - containerPort: 27017
        volumeMounts:
        - name: db-data
          mountPath: /data/db
  volumeClaimTemplates:
  - metadata:
      name: db-data
    spec:
      accessModes: [ "ReadWriteOnce" ]
      storageClassName: "manual"
      resources:
        requests:
          storage: 2Gi

但这不起作用 (mongo-stateful.yaml) pod 处于 pending 状态,正如我所描述的那样:

default-scheduler 0/3 个节点可用:1 个节点具有污染 {node-role.kubernetes.io/master: },该 pod 无法容忍,2 个 pod 具有未绑定的即时 PersistentVolumeClaims

PS:部署工作正常,没有任何错误,问题出在 Statefulset

有人可以帮我吗,如何用卷编写有状态集?

【问题讨论】:

    标签: mongodb kubernetes kubernetes-statefulset


    【解决方案1】:

    如果您的存储类不支持动态卷配置,您必须使用 yaml 文件手动创建 PV 和关联的 PVC,然后 volumeClaimTemplates 将允许将现有 PVC 与 statefulset 的 pod 链接。

    这是一个工作示例:https://github.com/k8s-school/k8s-school/blob/master/examples/MONGODB-install.sh

    你应该:

    • 在https://kind.sigs.k8s.io/本地运行它,它支持动态卷配置,所以这里会自动创建PVC和PV
    • 导出 PV 和 PVC yaml 文件
    • 使用这些 yaml 文件作为模板为您的 NFS 后端创建 PV 和 PVC。

    这是您将在 Kind 上获得的内容:

    $ ./MONGODB-install.sh               
    + kubectl apply -f 13-12-mongo-configmap.yaml
    configmap/mongo-init created
    + kubectl apply -f 13-11-mongo-service.yaml
    service/mongo created
    + kubectl apply -f 13-14-mongo-pvc.yaml
    statefulset.apps/mongo created
    $ kubectl get pods
    NAME      READY   STATUS    RESTARTS   AGE
    mongo-0   2/2     Running   0          8m38s
    mongo-1   2/2     Running   0          5m58s
    mongo-2   2/2     Running   0          5m45s
    $ kubectl get pvc
    NAME               STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
    database-mongo-0   Bound    pvc-05247511-096e-4af5-8944-17e0d8222512   1Gi        RWO            standard       8m42s
    database-mongo-1   Bound    pvc-f53c35a4-6fc0-4b18-b5fc-d7646815c0dd   1Gi        RWO            standard       6m2s
    database-mongo-2   Bound    pvc-2a711892-eeee-4481-94b7-6b46bf5b76a7   1Gi        RWO            standard       5m49s
    $ kubectl get pv 
    NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                      STORAGECLASS   REASON   AGE
    pvc-05247511-096e-4af5-8944-17e0d8222512   1Gi        RWO            Delete           Bound    default/database-mongo-0   standard                8m40s
    pvc-2a711892-eeee-4481-94b7-6b46bf5b76a7   1Gi        RWO            Delete           Bound    default/database-mongo-2   standard                5m47s
    pvc-f53c35a4-6fc0-4b18-b5fc-d7646815c0dd   1Gi        RWO            Delete           Bound    default/database-mongo-1   standard                6m1s
    

    还有一个 PVC 的转储(由 volumeClaimTemplate 在此处生成,因为 odf 类型的动态卷配置):

    $ kubectl get pvc database-mongo-0 -o yaml
    apiVersion: v1
    kind: PersistentVolumeClaim
    metadata:
      annotations:
        pv.kubernetes.io/bind-completed: "yes"
        pv.kubernetes.io/bound-by-controller: "yes"
        volume.beta.kubernetes.io/storage-provisioner: rancher.io/local-path
        volume.kubernetes.io/selected-node: kind-worker2
      creationTimestamp: "2020-10-16T15:05:20Z"
      finalizers:
      - kubernetes.io/pvc-protection
      labels:
        app: mongo
      managedFields:
        ...
      name: database-mongo-0
      namespace: default
      resourceVersion: "2259"
      selfLink: /api/v1/namespaces/default/persistentvolumeclaims/database-mongo-0
      uid: 05247511-096e-4af5-8944-17e0d8222512
    spec:
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 1Gi
      storageClassName: standard
      volumeMode: Filesystem
      volumeName: pvc-05247511-096e-4af5-8944-17e0d8222512
    status:
      accessModes:
      - ReadWriteOnce
      capacity:
        storage: 1Gi
      phase: Bound
    

    以及相关的PV:

    kubectl get pv pvc-05247511-096e-4af5-8944-17e0d8222512 -o yaml     
    apiVersion: v1
    kind: PersistentVolume
    metadata:
      annotations:
        pv.kubernetes.io/provisioned-by: rancher.io/local-path
      creationTimestamp: "2020-10-16T15:05:23Z"
      finalizers:
      - kubernetes.io/pv-protection
      managedFields:
        ...
      name: pvc-05247511-096e-4af5-8944-17e0d8222512
      resourceVersion: "2256"
      selfLink: /api/v1/persistentvolumes/pvc-05247511-096e-4af5-8944-17e0d8222512
      uid: 3d1e894e-0924-411a-8378-338e48ba4a28
    spec:
      accessModes:
      - ReadWriteOnce
      capacity:
        storage: 1Gi
      claimRef:
        apiVersion: v1
        kind: PersistentVolumeClaim
        name: database-mongo-0
        namespace: default
        resourceVersion: "2238"
        uid: 05247511-096e-4af5-8944-17e0d8222512
      hostPath:
        path: /var/local-path-provisioner/pvc-05247511-096e-4af5-8944-17e0d8222512_default_database-mongo-0
        type: DirectoryOrCreate
      nodeAffinity:
        required:
          nodeSelectorTerms:
          - matchExpressions:
            - key: kubernetes.io/hostname
              operator: In
              values:
              - kind-worker2
      persistentVolumeReclaimPolicy: Delete
      storageClassName: standard
      volumeMode: Filesystem
    status:
      phase: Bound
    

    【讨论】:

    • 我会看看,如果有任何问题,我会通知您。谢谢
    猜你喜欢
    • 1970-01-01
    • 2017-09-03
    • 1970-01-01
    • 2018-05-13
    • 1970-01-01
    • 2019-12-01
    • 2017-03-24
    • 1970-01-01
    • 2017-04-08
    相关资源
    最近更新 更多