k8s 在部署到 GKE 后终止旧 pod 时的停机时间

Question

有时（并非总是）当 Kubernetes 开始终止旧的 pod 时，我的应用程序会出现故障。它返回 HTTP 500 状态，因为 PHP 无法连接到 SQL Proxy 的 sidecar 容器。

这一切现在发生的频率降低了，但我注意到它仍然存在。我最后的尝试是将 livenessProbe 和 readinessProbe 添加到容器中，并包含一个 preStop 钩子以使 Apache 优雅地退出。再加上添加一个大的终止GracePeriodSeconds 并将-term_timeout=2000s 参数添加到 Cloud SQL 代理，试图让它再存活几分钟。

这一切难道不应该让旧豆荚平静地离开吗？还缺少什么？我的 k8s 清单如下：

apiVersion: apps/v1
kind: Deployment
metadata:
  name: myapp
spec:
  replicas: 3
  minReadySeconds: 60
  selector:
    matchLabels:
      run: myapp
  template:
    metadata:
      labels:
        run: myapp
    spec:
      readinessGates:
        - conditionType: "cloud.google.com/load-balancer-neg-ready"
      terminationGracePeriodSeconds: 2000
      containers:
      - name: myapp
        image: [my image URL goes here]
        lifecycle:
          preStop:
            exec:
              command: ["sh", "-c", "sleep 11 && /usr/sbin/apachectl graceful-stop" ]
        resources:
          requests:
            memory: 1Gi
            cpu: 500m
          limits:
            memory: 1Gi
            cpu: 500m
        ports:
        - containerPort: 80
        startupProbe:
          httpGet:
            path: /admin/
            port: 80
        readinessProbe:
          httpGet:
            path: /admin/
            port: 80
        livenessProbe:
          httpGet:
              path: /admin/
              port: 80
      - name: cloud-sql-proxy
        image: gcr.io/cloudsql-docker/gce-proxy:1.27.1-alpine
        command:
          - "/cloud_sql_proxy"
          - "-instances=myproject-development:southamerica-east1:mysql8-testing=tcp:3306"
          - "-term_timeout=2000s"
        securityContext:
          runAsNonRoot: true
        startupProbe:
            exec:
              command: ["nc", "-z", "127.0.0.1", "3306"]
        readinessProbe:
            exec:
              command: ["nc", "-z", "127.0.0.1", "3306"]
        livenessProbe:
            exec:
              command: ["nc", "-z", "127.0.0.1", "3306"]
        resources:
          requests:
            memory: "2Gi"
            cpu:    "1"
---
apiVersion: "autoscaling/v2beta1"
kind: "HorizontalPodAutoscaler"
metadata:
  name: "myapp-hpa"
  namespace: "default"
  labels:
    run: "myapp"
spec:
  scaleTargetRef:
    kind: "Deployment"
    name: "myapp"
    apiVersion: "apps/v1"
  minReplicas: 1
  maxReplicas: 7
  metrics:
  - type: "Resource"
    resource:
      name: "memory"
      targetAverageUtilization: 80
---
apiVersion: v1
kind: Service
metadata:
  name: myapp-service
  annotations:
    cloud.google.com/neg: '{"exposed_ports": {"80":{"name": "myapp-neg"}}}'
spec:
  type: ClusterIP
  ports:
  - port: 80
    targetPort: 80
  selector:
    run: myapp

Answer 1

您应该添加periodSeconds：执行探测的频率（以秒为单位）。默认为 10 秒。最小值为 1。

因此，如果您将periodSeconds 设置为非常少，如1，K8s 将每 1 秒检查一次 POD 的状态，非常频繁，您可能不会得到 500 响应。

一旦应用程序在端点停止提供 200，K8s 就会将 POD 标记为未就绪，并且不会有更多流量路由到它。