在 2 Namespace 中的 Cluster Kubernetes 中配置 Ingress-Nginx答案

【问题标题】：Configure Ingress-Nginix in Cluster Kubernetes in 2 Namespace在 2 Namespace 中的 Cluster Kubernetes 中配置 Ingress-Nginx
【发布时间】：2022-08-02 15:59:46
【问题描述】：

下午好

我正在使用 ingress-nginx 在本地 kubernetes 集群中进行服务公开。在这个集群中，我们管理 2 个环境：开发 (DEV) 和质量 (QA)。

我们想要的是以某种方式为每个环境（DEV 和 QA）拥有 1 个 ingress-nginx，但到目前为止我还无法配置它，我正在应用以下配置，但我无法为控制器中指示的 IP 执行此操作在请求之间根据环境，例如：

开发环境

控制器-部署-dev.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.15
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.1
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-dev
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx-dev
      app.kubernetes.io/instance: ingress-nginx-dev
      app.kubernetes.io/component: controller-dev
  revisionHistoryLimit: 10
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx-dev
        app.kubernetes.io/instance: ingress-nginx-dev
        app.kubernetes.io/component: controller-dev
    spec:
      dnsPolicy: ClusterFirst
      imagePullSecrets:
      - name: regcred
      containers:
        - name: controller
          image: 10.164.7.203:37003/tmve/ingress-nginx/controller:v1.1.1
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
            - --default-ssl-certificate=develop/srvdevma1-ssl
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
          volumeMounts:
            - name: webhook-cert
              mountPath: /usr/local/certificates/
              readOnly: true
          resources:
            requests:
              cpu: 1
              memory: 512Mi
      nodeSelector:
        kubernetes.io/hostname: tcold016 
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission

控制器-svc-dev.yaml

    apiVersion: v1
    kind: Service
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx
        app.kubernetes.io/instance: ingress-nginx
        app.kubernetes.io/version: 1.1.1
        app.kubernetes.io/component: controller
      name: ingress-nginx-controller-dev
      annotations:
        metallb.universe.tf/allow-shared-ip: shared-ip
      namespace: ingress-nginx
    spec:
      externalTrafficPolicy: Cluster
      loadBalancerIP: 10.161.169.12
      type: LoadBalancer
      ports:
        - name: http
          port: 80
          protocol: TCP
          targetPort: http
          appProtocol: http
        - name: https
          port: 30000
          protocol: TCP
          targetPort: https
          appProtocol: https
      selector:
        app.kubernetes.io/name: ingress-nginx-dev
        app.kubernetes.io/instance: ingress-nginx-dev
        app.kubernetes.io/component: controller-dev

规则入口开发

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-develop
  namespace: develop
  annotations: 
    nginx.ingress.kubernetes.io/backend-protocol: \"HTTPS\"
spec:
  ingressClassName: nginx
  tls:
    - secretName: srvdevma1-ssl
  rules:
  - http:
      paths:
      - path: /api/FindComplementaryAccountInfo
        pathType: Prefix
        backend:
          service:
            name: find-complementary-account-info
            port:
              number: 8083
      - path: /api/FindLimitedPackageBS
        pathType: Prefix
        backend:
          service:
            name: find-limited-package
            port:
              number: 8082             
      - path: /api/SendSMSBS
        pathType: Prefix
        backend:
          service: 
            name: send-sms
            port: 
              number: 8084              
      - path: /api/SubscribeLimitedPackageCS
        pathType: Prefix
        backend:
          service: 
            name: subscribe-limited-package
            port: 
              number: 8085

要在开发环境中使用服务，我们使用 controller-deployment-dev 中指示的 ip 和端口 30000

https://10.161.169.12:30000/api/FindLimitedPackageBS https://10.161.169.12:30000/api/FindComplementaryAccountInfo

质量保证环境

对于质量环境我有如下配置，和develop很相似，只是IP不同：

控制器-部署-qa.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    helm.sh/chart: ingress-nginx-4.0.15
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.1
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-tcold
  namespace: ingress-nginx
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: ingress-nginx-qa
      app.kubernetes.io/instance: ingress-nginx-qa
      app.kubernetes.io/component: controller-qa
  revisionHistoryLimit: 10
  minReadySeconds: 0
  template:
    metadata:
      labels:
        app.kubernetes.io/name: ingress-nginx-qa
        app.kubernetes.io/instance: ingress-nginx-qa
        app.kubernetes.io/component: controller-qa
    spec:
      dnsPolicy: ClusterFirst
      imagePullSecrets:
      - name: regcred
      containers:
        - name: controller
          image: 10.164.7.203:37003/tmve/ingress-nginx/controller:v1.1.1
          imagePullPolicy: IfNotPresent
          lifecycle:
            preStop:
              exec:
                command:
                  - /wait-shutdown
          args:
            - /nginx-ingress-controller
            - --election-id=ingress-controller-leader
            - --controller-class=k8s.io/ingress-nginx
            - --configmap=$(POD_NAMESPACE)/ingress-nginx-controller
            - --validating-webhook=:8443
            - --validating-webhook-certificate=/usr/local/certificates/cert
            - --validating-webhook-key=/usr/local/certificates/key
            - --default-ssl-certificate=develop/srvdevma1-ssl
          securityContext:
            capabilities:
              drop:
                - ALL
              add:
                - NET_BIND_SERVICE
            runAsUser: 101
            allowPrivilegeEscalation: true
          env:
            - name: POD_NAME
              valueFrom:
                fieldRef:
                  fieldPath: metadata.name
            - name: POD_NAMESPACE
              valueFrom:
                fieldRef:
                  fieldPath: metadata.namespace
            - name: LD_PRELOAD
              value: /usr/local/lib/libmimalloc.so
          livenessProbe:
            failureThreshold: 5
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          readinessProbe:
            failureThreshold: 3
            httpGet:
              path: /healthz
              port: 10254
              scheme: HTTP
            initialDelaySeconds: 10
            periodSeconds: 10
            successThreshold: 1
            timeoutSeconds: 1
          ports:
            - name: http
              containerPort: 80
              protocol: TCP
            - name: https
              containerPort: 443
              protocol: TCP
            - name: webhook
              containerPort: 8443
              protocol: TCP
          volumeMounts:
            - name: webhook-cert
              mountPath: /usr/local/certificates/
              readOnly: true
          resources:
            requests:
              cpu: 1
              memory: 512Mi
      nodeSelector:
        kubernetes.io/hostname: tcolt022
      serviceAccountName: ingress-nginx
      terminationGracePeriodSeconds: 300
      volumes:
        - name: webhook-cert
          secret:
            secretName: ingress-nginx-admission

控制器-svc-qa.yaml

apiVersion: v1
kind: Service
metadata:
  labels:
    app.kubernetes.io/name: ingress-nginx
    app.kubernetes.io/instance: ingress-nginx
    app.kubernetes.io/version: 1.1.1
    app.kubernetes.io/component: controller
  name: ingress-nginx-controller-qa
  annotations:
    metallb.universe.tf/allow-shared-ip: shared-ip
  namespace: ingress-nginx
spec:
  externalTrafficPolicy: Cluster
  loadBalancerIP: 10.161.173.45
  type: LoadBalancer
  ports:
    - name: http
      port: 80
      protocol: TCP
      targetPort: http
      appProtocol: http
    - name: https
      port: 30000
      protocol: TCP
      targetPort: https
      appProtocol: https
  selector:
    app.kubernetes.io/name: ingress-nginx-qa
    app.kubernetes.io/instance: ingress-nginx-qa
    app.kubernetes.io/component: controller-qa

规则入口 QA

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-calidad
  namespace: calidad
  annotations: 
    nginx.ingress.kubernetes.io/backend-protocol: \"HTTPS\"
spec:
  ingressClassName: nginx
  tls:
    - secretName: srvdevma1-ssl
  rules:
  - http:
      paths:
      - path: /api/FindComplementaryAccountInfo
        pathType: Prefix
        backend:
          service:
            name: find-complementary-account-info
            port:
              number: 8083
      - path: /api/FindLimitedPackageBS
        pathType: Prefix
        backend:
          service:
            name: find-limited-package
            port:
              number: 8082             
      - path: /api/SendSMSBS
        pathType: Prefix
        backend:
          service: 
            name: send-sms
            port: 
              number: 8084              
      - path: /api/SubscribeLimitedPackageCS
        pathType: Prefix
        backend:
          service: 
            name: subscribe-limited-package
            port: 
              number: 8085

所以你应该能够在这个环境中咨询服务，关于开发你应该只改变IP：

https://10.161.173.45:30000/api/FindLimitedPackageBS https://10.161.173.45:30000/api/FindComplementaryAccountInfo

有什么方法可以做我通过 ingress-nginx 指示的事情，条件是需要为服务维护相同的规则但在不同的命名空间中

更新

我设法通过以下非常好的文档找到了解决方案：

https://kubernetes.github.io/ingress-nginx/user-guide/multiple-ingress/

标签： kubernetes kubernetes-ingress nginx-ingress

【解决方案1】：

您可以通过使用 Ingress Classes 来实现此用例。入口可以由不同的控制器实现，通常具有不同的配置。每个 Ingress 都应该指定一个类，一个对 IngressClass 资源的引用，该资源包含额外的配置，包括应该实现该类的控制器的名称。

您可以通过授予它们对两个不同 IngressClass 的控制权来部署两个 Ingress 控制器，然后选择具有 ingressClassName 的两个 IngressClass 之一。确保 --controller-class= 和 --ingress-class 在每个入口控制器上设置为不同的值。

首先，指定'--controller-class=k8s.io/internal-ingress-nginx'和'--ingress-class=k8s.io/internal-nginx'在 ingress-nginx 部署中。然后在 IngressClass 中使用相同的 controller-class 值。并使用ingressClassName 在您的 Ingress 中引用该 IngressClass。有关如何设置 Ingress 类的更多信息，请参阅 Multiple Ingress Controllers。

笔记：如果--controller-class 设置为默认值k8s.io/ingress-nginx，控制器将监控没有类注解的 Ingress 和注解类设置为 nginx 的 Ingress。为 --controller-class 使用非默认值，以确保控制器仅满足特定的 Ingress 类。

您可以将 Ingress Controller 配置为仅处理来自特定命名空间的配置资源，该命名空间通过-watch-namespace命令行参数。如果您想为不同的应用程序使用不同的 NGINX 入口控制器，这将很有用，无论是在隔离和/或操作方面。

-watch-namespace用于监视 Ingress 资源的 Namespace。默认情况下，入口控制器监视所有命名空间。

请参阅Running Multiple Ingress Controllers 了解更多信息。

【讨论】：