Kubernetes 中的入口 nginx 路由（使用 nginx 反向代理从 docker-compose 转换）

Question

我正在尝试从在 docker-compose 中运行一些服务转移到 kubernetes，并努力从 nginx 反向代理转移到入口 nginx。该服务使用具有 Web 界面的h2o ai。我在 k8s 版本中无法调出网页界面。

我认为我需要修改入口文件中的规范以适当路由，感谢任何有关如何执行此操作的指示。

在使用 nginx 作为反向代理的 docker 解决方案中，当我访问 http://k8s-master:3002 时，它会重定向到 http://k8s-master:3002/flow/index.html 并正确显示

对于 k8s，（我在 32000 上以 NodePort 运行 ingress-nginx-controller），我尝试访问 https://k8s-master:32000/h2otest 并收到如下错误消息：

{"__meta":{"schema_version":3,"schema_name":"H2OErrorV3","schema_type":"H2OError"},"timestamp":1602674406013,"error_url":"Resource /h2otest","msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","dev_msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","http_status":404,"values":{},"exception_type":"water.exceptions.H2ONotFoundArgumentException","exception_msg":"\n\nERROR MESSAGE:\n\nResource /h2otest not found\n\n","stacktrace":["water.exceptions.H2ONotFoundArgumentException: Resource /h2otest not found","    water.api.RequestServer.response404(RequestServer.java:743)","    water.api.RequestServer.getResource(RequestServer.java:846)","    water.api.RequestServer.serve(RequestServer.java:465)","    water.api.RequestServer.doGeneric(RequestServer.java:301)","    water.api.RequestServer.doGet(RequestServer.java:225)","    javax.servlet.http.HttpServlet.service(HttpServlet.java:687)","    javax.servlet.http.HttpServlet.service(HttpServlet.java:790)","    org.eclipse.jetty.servlet.ServletHolder.handle(ServletHolder.java:865)","    org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:535)","    org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)","    org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)","    org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)","    org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)","    org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)","    org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)","    org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)","    org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)","    org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)","    water.webserver.jetty9.Jetty9ServerAdapter$LoginHandler.handle(Jetty9ServerAdapter.java:130)","    org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)","    org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)","    org.eclipse.jetty.server.Server.handle(Server.java:531)","    org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)","    org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)","    org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)","    org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)","    org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)","    org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)","    org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)","    java.lang.Thread.run(Thread.java:748)"]}

---

更新：基于 Ken Chen 的评论

我已将下面的入口更改为使用nginx.ingress.kubernetes.io/rewrite-target: /$2

现在，当我输入 https://k8s-master:32000/h2otest 时，它会重定向到：https://k8s-master:32000/flow/index.html

直接输入https://k8s-master:32000/h2otest/flow/index.html 即可。 如何修改规则以使 https://k8s-master:32000/h2otest 直接转到 https://k8s-master:32000/h2otest/flow/index.html？

---

以下包含每个版本的文件：

码头工人版本

docker-compose.yml

services :
  test-h2o-svc:
    build:
      context: h2o
      dockerfile: Dockerfile
    image: test-h2o:3.30.1.1
    restart: always
    networks:
      - testnet
#    ports:
#      - 54323:54321

  test-reverse-proxy:
    image: nginx:alpine
    hostname: reverse-proxy
    restart: always
    ports:
      - 3002:3002
    volumes:
      - "./nginx_files/nginx.conf:/etc/nginx/nginx.conf:rw"
      - "./nginx_files/sites-available/test.conf:/etc/nginx/conf.d/sites-available/test.conf:rw"
      - "./nginx_files/sites-available/test.conf:/etc/nginx/conf.d/sites-enabled/test.conf:rw"
    networks:
      - testnet

networks:
 testnet:
version: "3.5"

nginx.conf

user  nginx;
worker_processes  4;

error_log  /var/log/nginx/error.log warn;
pid        /var/run/nginx.pid;


events {
    worker_connections  1024;
}



http {
    include       /etc/nginx/mime.types;
    default_type  application/octet-stream;

    log_format  main  '$remote_addr - $remote_user [$time_local] "$request" '
                      '$status $body_bytes_sent "$http_referer" '
                      '"$http_user_agent" "$http_x_forwarded_for"';

    access_log  /var/log/nginx/access.log  main;

    sendfile        on;
    #tcp_nopush     on;
    keepalive_timeout  180;
    #gzip  on;

    include /etc/nginx/conf.d/sites-enabled/*.conf;
    client_body_buffer_size     50M;
    client_max_body_size 2000M;
}

test.conf

upstream link_h2o-svc {
  server        test-h2o-svc:54321;
}

server {
  listen        3002;
  location / {
    proxy_pass  http://link_h2o-svc;
   }
}

Dockerfile 用于容器：test-h2o:3.30.1.1

FROM ubuntu:latest

ARG H2O_VERSION=3.30.1.1

RUN apt-get update \
    && apt-get install default-jdk unzip wget -y

RUN wget https://h2o-release.s3.amazonaws.com/h2o/rel-zeno/1/h2o-${H2O_VERSION}.zip \
    && unzip h2o-${H2O_VERSION}.zip

ENV H2O_VERSION ${H2O_VERSION}
CMD java -jar h2o-${H2O_VERSION}/h2o.jar

---

k8s版本

kind: Namespace
apiVersion: v1
metadata:
  name: ns-test
  labels:
    name: ns-test
---
apiVersion: apps/v1
kind: Deployment
metadata:
  annotations:
    kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
    kompose.version: 1.21.0 (992df58d8)
  creationTimestamp: null
  labels:
    app: test-h2o
  name: test-h2o
  namespace: ns-test
spec:
  replicas: 1
  selector:
    matchLabels:
      app: test-h2o
  
  strategy: {}
  template:
    metadata:
      annotations:
        kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
        kompose.version: 1.21.0 (992df58d8)
      creationTimestamp: null
      labels:
        app: test-h2o
    spec:
      containers:
      - image: test-h2o:3.30.1.1
        imagePullPolicy: ""
        name: test-h2o
        resources: {}
      restartPolicy: Always
      serviceAccountName: ""
      volumes: null
status: {}
---
apiVersion: v1
kind: Service
metadata:
  annotations:
    kompose.cmd: /snap/kompose/19/kompose-linux-amd64 convert
    kompose.version: 1.21.0 (992df58d8)
  creationTimestamp: null

  name: test-h2o-svc
  namespace: ns-test
spec:
  ports:
  - name: "54321"
    port: 54321
    targetPort: 54321
  selector:
    app: test-h2o
status:
  loadBalancer: {}
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: ns-test
  annotations:
    #ingress.kubernetes.io/rewrite-target: /
    nginx.ingress.kubernetes.io/rewrite-target: /$2
spec:
  rules:
  - http:
      paths:
        - path: /h2otest(/|$)(.*)
          pathType: Prefix
          backend:
            service:
              name: test-h2o-svc
              port: 
                number: 54321

Answer 1

如何修改规则，使 https://k8s-master:30439/h2otest 直接转到 https://k8s-master:30439/h2otest/flow/index.html

这不是一个 nginx-ingress 问题。像大多数其他入口一样，nginx-ingress 可以通过 SSL 卸载来执行基于名称的虚拟主机和 SNI（或基于 SSL 名称的虚拟主机）。开箱即用，它不再受到限制。

虽然 nginx-ingress 可以进行一些基本的重写，但仅限于将 /path/(.*) 重写为 /(.*) 并将其传递给下一个容器以处理来自那里的流量。如果您希望它做更多事情，您需要为 nginx-ingress 编写配置映射。 是的，您需要为客户端正文大小等设置注释，但应用程序本身（不是入口）需要处理 /h2otest -> /h2otest/flow/index.html 重写（或 / -> /flow/index.html）。

根据您所描述的布局，您的布局看起来像 (nginx)ingress -> app

您可能需要将布局更改为(nginx)ingress -> nginx ->app，中间的 nginx 执行 / -> /flow/index.html 重写

忠告。如果您有任何机会使用基于名称的虚拟主机而不是路径虚拟主机（除非您有充分的理由，例如 cors）来基于路径。现在很多应用程序只是假设它们在/ 上，如果你在其他地方，配置会变得“有趣”

Answer 2

Nginx 反向代理将/h2otest 重写为/，但h2o 将/ 重定向为/flow/index.html。此时nginx没有处理/flow的规则所以结果为404。

尝试删除路径并改用 root /。

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: test-ingress
  namespace: ns-test
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
spec:
  rules:
  - http:
      paths:
        - path: /
          pathType: Prefix
          backend:
            serviceName: test-h2o-svc
            servicePort: 54321