Nginx 和 Ingress 与 Kubernetes 没有路由我的请求

Question

我有 Docker、Kubernetes(1.7) 和 Nginx 都在我的 RHEL7 服务器上运行，我自己的服务位于 docker 容器内并被 Kubernetes 接收。我知道 Kubernetes 与 docker 配合得很好，因为我可以使用它自己的 IP:PORT 地址调用 Kubernetes pod 的 get 请求并且它可以工作。我使用默认后端设置 Nginx 并让所有这些工作。我通过调用get pods 和get svc 命令知道这一点，并且一切都在正常运行。当我创建入口时，我知道 Nginx 正在接收它，因为当我使用命令 kubectl describe pods {NGNIX-CONTROLLER} 时，我看到它更新了入口，甚至记录了我命名的内容。现在，我使用kubectl clusterinfo 获取 Kubernetes 主服务器的 IP 地址，并使用此 IP 地址尝试调用我的服务，类似于http://KUBEIPADDRESS/PATH/TO/MY/SERVICE，没有端口号，但它不起作用。我不知道发生了什么。有人可以帮我为什么 Ingress 和/或 Nnginx 没有正确路由到我的服务吗？我将在下面给出我的入口和 nginx 文件。

（注意，对于nginx yaml文件，nginx控制器的部署一路在底层。）

入口yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: gateway-ingress
  annotations:
    kubernetes.io/ingress.class: nginx
    ingress.kubernetes.io/rewrite-target: /
spec:
  backend:
    serviceName: default-http-backend
    servicePort: 80
  rules:
  - host: testhost
    http:
      paths:
      - path: /customer
        backend:
          serviceName: customer
          servicePort: 9001

nginx 控制器 yaml

apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRole
metadata:
  name: ingress
rules:
- apiGroups:
  - ""
  - "extensions"
  resources:
  - configmaps
  - secrets
  - services
  - endpoints
  - ingresses
  - nodes
  - pods
  verbs:
  - list
  - watch
- apiGroups:
  - "extensions"
  resources:
  - ingresses
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - events
  - services
  verbs:
  - create
  - list
  - update
  - get
- apiGroups:
  - "extensions"
  resources:
  - ingresses/status
  - ingresses
  verbs:
  - update
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: Role
metadata:
  name: ingress-ns
  namespace: kube-system
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - list
- apiGroups:
  - ""
  resources:
  - services
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - endpoints
  verbs:
  - get
  - create
  - update  
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: ingress-ns-binding
  namespace: kube-system
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: ingress-ns
subjects:
  - kind: ServiceAccount
    name: ingress
    namespace: kube-system
---
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: ClusterRoleBinding
metadata:
  name: ingress-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: ingress
subjects:
  - kind: ServiceAccount
    name: ingress
    namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: default-http-backend
  labels:
    k8s-app: default-http-backend
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: default-http-backend
    spec:
      terminationGracePeriodSeconds: 60
      containers:
      - name: default-http-backend
        # Any image is permissable as long as:
        # 1. It serves a 404 page at /
        # 2. It serves 200 on a /healthz endpoint
        image: gcr.io/google_containers/defaultbackend:1.0
        livenessProbe:
          httpGet:
            path: /healthz
            port: 8080
            scheme: HTTP
          initialDelaySeconds: 30
          timeoutSeconds: 5
        ports:
        - containerPort: 8080
        resources:
          limits:
            cpu: 10m
            memory: 20Mi
          requests:
            cpu: 10m
            memory: 20Mi
---
apiVersion: v1
kind: Service
metadata:
  name: default-http-backend
  namespace: kube-system
  labels:
    k8s-app: default-http-backend
spec:
  ports:
  - port: 80
    targetPort: 8080
  selector:
    k8s-app: default-http-backend
---
apiVersion: v1
kind: ServiceAccount
metadata:
  name: ingress
  namespace: kube-system
---
apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: nginx-ingress-controller
  labels:
    k8s-app: nginx-ingress-controller
  namespace: kube-system
spec:
  replicas: 1
  template:
    metadata:
      labels:
        k8s-app: nginx-ingress-controller
    spec:
      # hostNetwork makes it possible to use ipv6 and to preserve the source IP correctly regardless of docker configuration
      # however, it is not a hard dependency of the nginx-ingress-controller itself and it may cause issues if port 10254 already is taken on the host
      # that said, since hostPort is broken on CNI (https://github.com/kubernetes/kubernetes/issues/31307) we have to use hostNetwork where CNI is used
      # like with kubeadm
      hostNetwork: true
      terminationGracePeriodSeconds: 60
      serviceAccountName: ingress
      containers:
      - image: gcr.io/google_containers/nginx-ingress-controller:0.9.0-beta.3
        name: nginx-ingress-controller
        readinessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
        livenessProbe:
          httpGet:
            path: /healthz
            port: 10254
            scheme: HTTP
          initialDelaySeconds: 10
          timeoutSeconds: 1
        ports:
        - containerPort: 80
          hostPort: 80
        - containerPort: 443
          hostPort: 443
        env:
          - name: POD_NAME
            valueFrom:
              fieldRef:
                fieldPath: metadata.name
          - name: POD_NAMESPACE
            valueFrom:
              fieldRef:
                fieldPath: metadata.namespace
        args:
        - /nginx-ingress-controller
        - --default-backend-service=$(POD_NAMESPACE)/default-http-backend

当我做kubectl describe ing 时，我得到了

Name:                   gateway-ingress
Namespace:              default
Address:
Default backend:        default-http-backend:80 (<none>)
Rules:
  Host          Path    Backends
  ----          ----    --------
  testhost

                /customer    customer:9001 ({IP}:9001,{IP}:9001)
Annotations:
  rewrite-target:       /
Events:                 <none>

这是我为客户提供的部署和服务，以防有人需要

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  name: customer
  labels:
    run: customer
spec:
  replicas: 2
  template:
    metadata:
      labels:
        run: customer
    spec:
      containers:
      - name: customer
        image: customer
        imagePullPolicy: Always
        ports:
        - containerPort: 9001
          protocol: TCP
---
kind: Service
apiVersion: v1
metadata:
  name: customer
spec:
  selector:
    run: customer
  type: NodePort
  ports:
  - name: port1
    protocol: TCP
    port: 9001
    targetPort: 9001

Answer 1

据我所知，您的设置存在一些问题：

• KUBEIPADDRESS 在您调用的 URL 中：IP 地址将不起作用，因为您将 Ingress 配置为侦听 testhost。所以你需要调用http://testhost/customer，并配置你的网络以将testhost解析为正确的IP地址

• 但是正确的 IP 地址是什么？您正在尝试在端口 80 上使用 k8s master。如果没有进一步的配置，这将无法工作。为此，您需要为 Ingress Controller 使用 NodePort 服务，它将其暴露在端口 80（可能是 433）上。为了使用这些低端口，您需要通过 kube-apiserver 选项来允许它，请参阅 https://kubernetes.io/docs/admin/kube-apiserver/ 上的 --service-node-port-range。一旦成功，您可以将 k8s 集群的任何节点的任何 IP 地址用于testhost。注意：确保没有其他应用程序在任何节点上使用这些端口！