使用 GCP 网络端点组进行容器原生负载平衡的不健康后端

Question

我们正在测试 google 的新 container-native load balancingfeature。我们成功地关注了这个tutorial，我们正在尝试将它推广到我们在 GKE 上的三个服务中。

据我所知，NEG 功能和旧版 GCLB 入口对象之间的唯一区别是每个服务中的注释，因此 URL 映射应该工作相同。

我们已更新所有服务以使用此注释，但三分之二是 Unhealthy，而一个被认为是健康的。服务 yaml 的唯一区别是名称和选择器。

所有部署都有健康检查，当我们手动检查时都是健康的，但 LB 说后端不健康。

我们缺少什么？

Ingress.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: fanout-ingress
  annotations:
    kubernetes.io/ingress.global-static-ip-name: "neg-ip"
spec:
  backend:
    serviceName: frontend-svc
    servicePort: 8080
  rules:
  - host: testneg.test.com
    http:
      paths:
      - path: /*
        backend:
          serviceName: frontend-svc # Healthy service
          servicePort: 8080
      - path: /backend/*
        backend:
          serviceName: backend-svc # Unhealthy service
          servicePort: 8080
      - path: /notifications/*
        backend:
          serviceName: notifications-svc # Unhealthy service
          servicePort: 8080

--

frontend-svc.yaml - 除了名称和选择器之外，后端/通知是相同的

apiVersion: v1
kind: Service
metadata:
  name: frontend-svc
  annotations:
    cloud.google.com/neg: '{"ingress": true}' # Creates an NEG after an Ingress is created
spec:
  selector:
    app: frontend
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080

--

backend-deployment.yaml

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: backend
spec:
  replicas: 1
  minReadySeconds: 60
  strategy:
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 1
    type: RollingUpdate
  template:
    spec:
      containers:
        image: us.gcr.io/<OUR_DJANGO_IMAGE>
        imagePullPolicy: Always
        name: backend
        ports:
        - containerPort: 8080
          protocol: TCP
        terminationMessagePath: /dev/termination-log
        terminationMessagePolicy: File
        readinessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 30
          timeoutSeconds: 3
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 60
          periodSeconds: 30
          timeoutSeconds: 3   
      terminationGracePeriodSeconds: 60

Answer 1

您的入口 yaml 文件显示不同的服务

- path: /*
        backend:
          serviceName: frontend-svc # Healthy service
          servicePort: 8080
      - path: /backend/*
        backend:
          serviceName: backend-svc # Unhealthy service
          servicePort: 8080
      - path: /notifications/*
        backend:
          serviceName: notifications-svc # Unhealthy service
          servicePort: 8080

您的 frontend-svc.yaml 有一个不同的服务名称“li-frontend-svc”，它不在您的入口中。

Ingress 中的 Spec.Backend.serviceName 应与您的服务名称匹配，预计后端服务不健康。

上次编辑：

在您的入口中，您指定两次服务前端-svc，您应该使用入口规范，如下规范：

spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: first-service # Name of the Service targeted by the Ingress
          servicePort: 8080 # Should match the port used by the Service
        path: <first-service-path>/*
      - backend:
          serviceName: second-service # Name of the Service targeted by the Ingress
          servicePort: 8080 # Should match the port used by the Service
        path: <second-service-path>/*
      - backend:
          serviceName: third-service # Name of the Service targeted by the Ingress
          servicePort: 8080 # Should match the port used by the Service
        path: <third-service-path>/*

这是我的复制品：

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: neg-hello-1 # Label for the Deployment
  name: neg-hello-1 # Name of Deployment
spec: # Deployment's specification
  minReadySeconds: 60 # Number of seconds to wait after a Pod is created and its status is Ready
  selector:
    matchLabels:
      run: neg-hello-1
  template: # Pod template
    metadata:
      labels:
        run: neg-hello-1 # Labels Pods from this Deployment
    spec: # Pod specification; each Pod created by this Deployment has this specification
      containers:
      - image: gcr.io/google-samples/hello-app:1.0 # Application to run in Deployment's Pods
        name: neg-hello-1 # Container name
        ports:
        - containerPort: 8080 # Port used by containers running in these Pods
          protocol: TCP
        readinessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 15
          periodSeconds: 20
      terminationGracePeriodSeconds: 60 # Number of seconds to wait for connections to terminate before shutting down Pods

---

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: neg-hello-2 # Label for the Deployment
  name: neg-hello-2 # Name of Deployment
spec: # Deployment's specification
  minReadySeconds: 60 # Number of seconds to wait after a Pod is created and its status is Ready
  selector:
    matchLabels:
      run: neg-hello-2
  template: # Pod template
    metadata:
      labels:
        run: neg-hello-2 # Labels Pods from this Deployment
    spec: # Pod specification; each Pod created by this Deployment has this specification
      containers:
      - image: gcr.io/google-samples/hello-app:2.0 # Application to run in Deployment's Pods
        name: neg-hello-2 # Container name
        ports:
        - containerPort: 8080 # Port used by containers running in these Pods
          protocol: TCP
        readinessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 5
          periodSeconds: 10
        livenessProbe:
          tcpSocket:
            port: 8080
          initialDelaySeconds: 15
          periodSeconds: 20
      terminationGracePeriodSeconds: 60 # Number of seconds to wait for connections to terminate before shutting down Pods

--

apiVersion: v1
kind: Service
metadata:
  name: neg-hello-1 # Name of Service
  annotations:
    cloud.google.com/neg: '{"ingress": true}' # Creates an NEG after an Ingress is created
spec: # Service's specification
  selector:
    run: neg-hello-1 # Selects Pods labelled run: neg-hello-1
  ports:
  - port: 80 # Service's port
    protocol: TCP
    targetPort: 8080

--

apiVersion: v1
kind: Service
metadata:
  name: neg-hello-2 # Name of Service
  annotations:
    cloud.google.com/neg: '{"ingress": true}' # Creates an NEG after an Ingress is created
spec: # Service's specification
  selector:
    run: neg-hello-2 # Selects Pods labelled run: neg-hello-2
  ports:
  - port: 80 # Service's port
    protocol: TCP
    targetPort: 8080

--

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: neg-ingress
spec:
  rules:
  - http:
      paths:
      - backend:
          serviceName: neg-hello-1 # Name of the Service targeted by the Ingress
          servicePort: 80 # Should match the port used by the Service
        path: /*
      - backend:
          serviceName: neg-hello-2 # Name of the Service targeted by the Ingress
          servicePort: 80 # Should match the port used by the Service
        path: /v2/*