Microk8s + metallb + ingress

Question

我对 kubernetes 很陌生，我正在尝试在带有 CentOS 的 VPS 上设置 microk8s 测试环境。

我做了什么：

我设置了集群，启用了 ingress 和 metallb

microk8s enable ingress
microk8s enable metallb

暴露了入口控制器服务：

apiVersion: v1
kind: Service
metadata:
  name: ingress
  namespace: ingress
spec:
  type: LoadBalancer
  selector:
    name: nginx-ingress-microk8s 
  ports:
  - name: http
    protocol: TCP
    port: 80
    targetPort: 80
  - name: https
    protocol: TCP
    port: 443
    targetPort: 443

暴露一个nginx部署来测试入口

apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    run: nginx
  name: nginx-deploy
spec:
  replicas: 1
  selector:
    matchLabels:
      run: nginx-deploy
  template:
    metadata:
      labels:
        run: nginx-deploy
    spec:
      containers:
      - image: nginx
        name: nginx

这是我的集群的状态：

NAMESPACE            NAME                                             READY   STATUS    RESTARTS   AGE
kube-system          pod/hostpath-provisioner-5c65fbdb4f-m2xq6        1/1     Running   3          41h
kube-system          pod/coredns-86f78bb79c-7p8bs                     1/1     Running   3          41h
kube-system          pod/calico-node-g4ws4                            1/1     Running   6          42h
kube-system          pod/calico-kube-controllers-847c8c99d-xhmd7      1/1     Running   4          42h
kube-system          pod/metrics-server-8bbfb4bdb-ggvk7               1/1     Running   0          41h
kube-system          pod/kubernetes-dashboard-7ffd448895-ktv8j        1/1     Running   0          41h
kube-system          pod/dashboard-metrics-scraper-6c4568dc68-l4xmg   1/1     Running   0          41h
container-registry   pod/registry-9b57d9df8-xjh8d                     1/1     Running   0          38h
cert-manager         pod/cert-manager-cainjector-5c6cb79446-vv5j2     1/1     Running   0          12h
cert-manager         pod/cert-manager-794657589-srrmr                 1/1     Running   0          12h
cert-manager         pod/cert-manager-webhook-574c9758c9-9dwr6        1/1     Running   0          12h
metallb-system       pod/speaker-9gjng                                1/1     Running   0          97m
metallb-system       pod/controller-559b68bfd8-trk5z                  1/1     Running   0          97m
ingress              pod/nginx-ingress-microk8s-controller-f6cdb      1/1     Running   0          65m
default              pod/nginx-deploy-5797b88878-vgp7x                1/1     Running   0          20m

NAMESPACE            NAME                                TYPE           CLUSTER-IP       EXTERNAL-IP    PORT(S)                      AGE
default              service/kubernetes                  ClusterIP      10.152.183.1     <none>         443/TCP                      42h
kube-system          service/kube-dns                    ClusterIP      10.152.183.10    <none>         53/UDP,53/TCP,9153/TCP       41h
kube-system          service/metrics-server              ClusterIP      10.152.183.243   <none>         443/TCP                      41h
kube-system          service/kubernetes-dashboard        ClusterIP      10.152.183.225   <none>         443/TCP                      41h
kube-system          service/dashboard-metrics-scraper   ClusterIP      10.152.183.109   <none>         8000/TCP                     41h
container-registry   service/registry                    NodePort       10.152.183.44    <none>         5000:32000/TCP               38h
cert-manager         service/cert-manager                ClusterIP      10.152.183.183   <none>         9402/TCP                     12h
cert-manager         service/cert-manager-webhook        ClusterIP      10.152.183.99    <none>         443/TCP                      12h
echoserver           service/echoserver                  ClusterIP      10.152.183.110   <none>         80/TCP                       72m
ingress              service/ingress                     LoadBalancer   10.152.183.4     192.168.0.11   80:32617/TCP,443:31867/TCP   64m
default              service/nginx-deploy                ClusterIP      10.152.183.149   <none>         80/TCP                       19m

NAMESPACE        NAME                                               DESIRED   CURRENT   READY   UP-TO-DATE   AVAILABLE   NODE SELECTOR                 AGE
kube-system      daemonset.apps/calico-node                         1         1         1       1            1           kubernetes.io/os=linux        42h
metallb-system   daemonset.apps/speaker                             1         1         1       1            1           beta.kubernetes.io/os=linux   97m
ingress          daemonset.apps/nginx-ingress-microk8s-controller   1         1         1       1            1           <none>                        65m

NAMESPACE            NAME                                        READY   UP-TO-DATE   AVAILABLE   AGE
kube-system          deployment.apps/hostpath-provisioner        1/1     1            1           41h
kube-system          deployment.apps/coredns                     1/1     1            1           41h
kube-system          deployment.apps/calico-kube-controllers     1/1     1            1           42h
kube-system          deployment.apps/metrics-server              1/1     1            1           41h
kube-system          deployment.apps/dashboard-metrics-scraper   1/1     1            1           41h
kube-system          deployment.apps/kubernetes-dashboard        1/1     1            1           41h
container-registry   deployment.apps/registry                    1/1     1            1           38h
cert-manager         deployment.apps/cert-manager-cainjector     1/1     1            1           12h
cert-manager         deployment.apps/cert-manager                1/1     1            1           12h
cert-manager         deployment.apps/cert-manager-webhook        1/1     1            1           12h
metallb-system       deployment.apps/controller                  1/1     1            1           97m
default              deployment.apps/nginx-deploy                1/1     1            1           20m

NAMESPACE            NAME                                                   DESIRED   CURRENT   READY   AGE
kube-system          replicaset.apps/hostpath-provisioner-5c65fbdb4f        1         1         1       41h
kube-system          replicaset.apps/coredns-86f78bb79c                     1         1         1       41h
kube-system          replicaset.apps/calico-kube-controllers-847c8c99d      1         1         1       42h
kube-system          replicaset.apps/metrics-server-8bbfb4bdb               1         1         1       41h
kube-system          replicaset.apps/kubernetes-dashboard-7ffd448895        1         1         1       41h
kube-system          replicaset.apps/dashboard-metrics-scraper-6c4568dc68   1         1         1       41h
container-registry   replicaset.apps/registry-9b57d9df8                     1         1         1       38h
cert-manager         replicaset.apps/cert-manager-cainjector-5c6cb79446     1         1         1       12h
cert-manager         replicaset.apps/cert-manager-794657589                 1         1         1       12h
cert-manager         replicaset.apps/cert-manager-webhook-574c9758c9        1         1         1       12h
metallb-system       replicaset.apps/controller-559b68bfd8                  1         1         1       97m
default              replicaset.apps/nginx-deploy-5797b88878                1         1         1       20m

看起来 Metallb 可以正常工作，因为入口服务从我指定的池中收到了一个 ip。 现在，当我尝试部署入口以到达 nginx 部署时，我没有得到地址：

apiVersion: networking.k8s.io/v1beta1
kind: Ingress
metadata:
  annotations:
    kubernetes.io/ingress.class: nginx
    nginx.ingress.kubernetes.io/rewrite-target: /
  name: ingress-nginx-deploy
spec:
  rules:
  - host: test.com
    http:
      paths:
      - backend:
          serviceName: nginx-deploy
          servicePort: 80

NAMESPACE   NAME                   CLASS    HOSTS                       ADDRESS   PORTS   AGE
default     ingress-nginx-deploy   <none>   test.com                              80      13m

非常感谢您的帮助。谢谢！

Answer 1

TL;DR

有一些方法可以修复您的Ingress，以便它可以获取 IP 地址。

您可以任一：

• 删除kubernetes.io/ingress.class: nginx 并在spec 部分下添加ingressClassName: public。
• 使用官方文档中较新的示例 (apiVersion)，默认情况下将分配一个 IngressClass：
  • Kubernetes.io: Docs: Concepts: Services networking: Ingress

将解决您的问题的Ingress 资源示例：

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: ingress-nginx-deploy
spec:
  ingressClassName: public 
  # above field is optional as microk8s default ingressclass will be assigned
  rules:
  - host: test.com
    http:
      paths:
      - path: /
        pathType: Prefix
        backend:
          service:
            name: nginx-deploy
            port:
              number: 80

您可以通过以下官方文档了解更多关于IngressClass的信息：

• Kubernetes.io: Blog: Improvements to the Ingress API in Kubernetes 1.18

我已经包含了更多解释，应该对这个特定的设置有更多的了解。

---

在你申请以上Ingress资源后的输出：

• $ kubectl get ingress

将跟随：

NAME                   CLASS    HOSTS      ADDRESS     PORTS   AGE
ingress-nginx-deploy   public   test.com   127.0.0.1   80      43s

如您所见，ADDRESS 包含127.0.0.1。这是因为这个特定的Ingress controller 由插件启用，绑定到您的主机（MicroK8S 节点）到端口80,443。

运行可以看到：

• $ sudo microk8s kubectl get daemonset -n ingress nginx-ingress-microk8s-controller -o yaml

附注！

寻找hostPort 和securityContext.capabilities。

您创建的LoadBalancer 类型的Service 将与您的Ingress controller 一起使用，但不会显示在$ kubectl get ingress 中的ADDRESS 下。

附注！

请记住，在此特定设置中，您需要使用 Header Host: test.com 连接到您的 Ingress controller，除非您已配置 DNS 解析以支持您的设置。否则你会得到一个404。

---

其他资源：

• Github.com: Ubuntu: Microk8s: Ingress with MetalLb issues
• Kubernetes.io: Docs: Concepts: Configuration: Overview