无法从 Postman 的 k8s-Skaffold 中获得 Express API 的响应

Question

尝试做一些应该很简单的事情：启动一个 Express pod 并获取应该以 Hello World! 响应的 localhost:5000/。

• 我已经为Docker for Mac 和minikube 安装了ingress-nginx
  • 强制：kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
  • Docker for Mac：kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/cloud-generic.yaml
  • minikube：minikube addons enable ingress
• 我跑skaffold dev --tail
• 它打印出Example app listening on port 5000，显然正在运行
• 导航到 localhost 和 localhost:5000 并收到“无法获得任何响应”错误
• 另外，尝试minikube ip 即192.168.99.100 并获得相同的结果

不太确定我在这里做错了什么。代码和配置如下。有什么建议吗？

---

index.js

// Import dependencies
const express = require('express');

// Set the ExpressJS application
const app = express();

// Set the listening port
// Web front-end is running on port 3000
const port = 5000;

// Set root route
app.get('/', (req, res) => res.send('Hello World!'));

// Listen on the port
app.listen(port, () => console.log(`Example app listening on port ${port}`));

---

skaffold.yaml

apiVersion: skaffold/v1beta15
kind: Config
build:
  local:
    push: false
  artifacts:
    - image: sockpuppet/server
      context: server
      docker:
        dockerfile: Dockerfile.dev
      sync:
        manual:
        - src: '**/*.js'
          dest: .
deploy:
  kubectl:
    manifests:
      - k8s/ingress-service.yaml
      - k8s/server-deployment.yaml
      - k8s/server-cluster-ip-service.yaml

---

ingress-service.yaml

apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: ingress-service
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /$1
spec:
  rules:
    - http:
        paths:
          - path: /?(.*)
            backend:
              serviceName: server-cluster-ip-service
              servicePort: 5000

---

server-deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: server-deployment
spec:
  replicas: 3
  selector:
    matchLabels:
      component: server
  template:
    metadata:
      labels:
        component: server
    spec:
      containers:
        - name: server
          image: sockpuppet/server
          ports:
            - containerPort: 5000

---

server-cluster-ip-service.yaml

apiVersion: v1
kind: Service
metadata:
  name: server-cluster-ip-service
spec:
  type: ClusterIP
  selector:
    component: server
  ports:
    - port: 5000
      targetPort: 5000

---

Dockerfile.dev

FROM node:12.10-alpine
EXPOSE 5000

WORKDIR "/app"
COPY ./package.json ./
RUN npm install
COPY . .

CMD ["npm", "run", "dev"]

---

来自describe的输出

$ kubectl describe ingress ingress-service     
Name:             ingress-service
Namespace:        default
Address:          
Default backend:  default-http-backend:80 (<none>)
Rules:
  Host       Path  Backends
  ----       ----  --------
  localhost  
             /   server-cluster-ip-service:5000 (172.17.0.7:5000,172.17.0.8:5000,172.17.0.9:5000)
Annotations:
  kubectl.kubernetes.io/last-applied-configuration:  {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx"},"name":"ingress-service","namespace":"default"},"spec":{"rules":[{"host":"localhost","http":{"paths":[{"backend":{"serviceName":"server-cluster-ip-service","servicePort":5000},"path":"/"}]}}]}}

  kubernetes.io/ingress.class:  nginx
Events:
  Type    Reason  Age   From                      Message
  ----    ------  ----  ----                      -------
  Normal  CREATE  16h   nginx-ingress-controller  Ingress default/ingress-service
  Normal  CREATE  21s   nginx-ingress-controller  Ingress default/ingress-service

---

来自kubectl get po -l component=server的输出

$ kubectl get po -l component=server
NAME                                READY   STATUS    RESTARTS   AGE
server-deployment-cf6dd5744-2rnh9   1/1     Running   0          11s
server-deployment-cf6dd5744-j9qvn   1/1     Running   0          11s
server-deployment-cf6dd5744-nz4nj   1/1     Running   0          11s

来自kubectl describe pods server-deployment的输出：注意到Host Port: 0/TCP。可能是问题？

Name:               server-deployment-6b78885779-zttns
Namespace:          default
Priority:           0
PriorityClassName:  <none>
Node:               minikube/10.0.2.15
Start Time:         Tue, 08 Oct 2019 19:54:03 -0700
Labels:             app.kubernetes.io/managed-by=skaffold-v0.39.0
                    component=server
                    pod-template-hash=6b78885779
                    skaffold.dev/builder=local
                    skaffold.dev/cleanup=true
                    skaffold.dev/deployer=kubectl
                    skaffold.dev/docker-api-version=1.39
                    skaffold.dev/run-id=c545df44-a37d-4746-822d-392f42817108
                    skaffold.dev/tag-policy=git-commit
                    skaffold.dev/tail=true
Annotations:        <none>
Status:             Running
IP:                 172.17.0.5
Controlled By:      ReplicaSet/server-deployment-6b78885779
Containers:
  server:
    Container ID:   docker://2d0aba8f5f9c51a81f01acc767e863b7321658f0a3d0839745adb99eb0e3907a
    Image:          sockpuppet/server:668dfe550d93a0ae76eb07e0bab900f3968a7776f4f177c97f61b18a8b1677a7
    Image ID:       docker://sha256:668dfe550d93a0ae76eb07e0bab900f3968a7776f4f177c97f61b18a8b1677a7
    Port:           5000/TCP
    Host Port:      0/TCP
    State:          Running
      Started:      Tue, 08 Oct 2019 19:54:05 -0700
    Ready:          True
    Restart Count:  0
    Environment:    <none>
    Mounts:
      /var/run/secrets/kubernetes.io/serviceaccount from default-token-qz5kr (ro)
Conditions:
  Type              Status
  Initialized       True
  Ready             True
  ContainersReady   True
  PodScheduled      True
Volumes:
  default-token-qz5kr:
    Type:        Secret (a volume populated by a Secret)
    SecretName:  default-token-qz5kr
    Optional:    false
QoS Class:       BestEffort
Node-Selectors:  <none>
Tolerations:     node.kubernetes.io/not-ready:NoExecute for 300s
                 node.kubernetes.io/unreachable:NoExecute for 300s
Events:
  Type    Reason     Age        From               Message
  ----    ------     ----       ----               -------
  Normal  Scheduled  <unknown>  default-scheduler  Successfully assigned default/server-deployment-6b78885779-zttns to minikube
  Normal  Pulled     7s         kubelet, minikube  Container image "sockpuppet/server:668dfe550d93a0ae76eb07e0bab900f3968a7776f4f177c97f61b18a8b1677a7" already present on machine
  Normal  Created    7s         kubelet, minikube  Created container server
  Normal  Started    6s         kubelet, minikube  Started container server

Answer 1

好的，现在解决了。

归结为所使用的服务类型：ClusterIP。

ClusterIP：在集群内部 IP 上公开服务。选择此值使服务只能从集群内访问。这是默认的 ServiceType。

如果我想直接从集群外部（例如 Postman、pgAdmin 等）连接到 Pod 或 Deployment。并且我想使用 Service 来实现，我应该使用NodePort:

NodePort：在每个 Node 的 IP 上的静态端口（NodePort）上公开服务。将自动创建 NodePort 服务将路由到的 ClusterIP 服务。您可以通过请求 <NodeIP>:<NodePort> 从集群外部联系 NodePort 服务。

所以在我的情况下，如果我想继续使用服务，我会将我的服务清单更改为：

apiVersion: v1
kind: Service
metadata:
  name: server-cluster-ip-service
spec:
  type: NodePort
  selector:
    component: server
  ports:
    - port: 5000
      targetPort: 5000
      nodePort: 31515

确保手动设置nodePort: <port>，否则它有点随机且使用起来很痛苦。

然后我会使用minikube ip 获取minikube IP，并使用192.168.99.100:31515 连接到 Pod。

那时，一切都按预期进行。

但这意味着拥有单独的开发集 (NodePort) 和生产 (ClusterIP) 清单，这可能完全没问题。但我希望我的清单尽可能接近生产版本（即ClusterIP）。

有几种方法可以解决这个问题：

1. 使用 Kustomize 之类的东西，您可以在其中设置 base.yaml，然后为每个环境设置叠加层，它只是更改相关信息，避免清单中的大部分重复。

2. 使用kubectl port-forward。我想这就是我要走的路。这样我就可以保留我的一组生产清单，但是当我想用 pgAdmin 对 Postgres 进行 QA 时，我可以这样做：

   kubectl port-forward services/postgres-cluster-ip-service 5432:5432

   或者对于后端和邮递员：

   kubectl port-forward services/server-cluster-ip-service 5000:5000

我正在通过ingress-service.yaml 使用nginx-ingress 进行此操作，但还没有完成。当我这样做时会更新。但对我来说，port-forward 似乎是可行的方法，因为我可以只拥有一组无需更改的生产清单。

Skaffold 端口转发

这更符合我的需求。将此附加到skaffold.yaml 的底部，基本上与kubectl port-forward 相同，无需绑定一两个终端：

portForward:
  - resourceType: service
    resourceName: server-cluster-ip-service
    port: 5000
    localPort: 5000
  - resourceType: service
    resourceName: postgres-cluster-ip-service
    port: 5432
    localPort: 5432

然后运行skaffold dev --port-forward。