在 minkube 上带有 nginx 入口控制器的 Metallb 不断为入口重置外部 IP

Question

这里有一个 MCVE 示例：https://github.com/chrissound/k8s-metallb-nginx-ingress-minikube （只需运行 ./init.sh 和 minikube addons enable ingress）。

分配给入口的IP不断重置，我不知道是什么原因造成的？我可能需要额外的配置吗？

kubectl get ingress --all-namespaces                 
NAMESPACE       NAME          HOSTS         ADDRESS           PORTS     AGE
chris-example   app-ingress   example.com   192.168.122.253   80, 443   61m

一分钟后：

NAMESPACE       NAME          HOSTS         ADDRESS   PORTS     AGE
chris-example   app-ingress   example.com             80, 443   60m

---

就我刚刚申请的配置而言：

# metallb
kubectl apply -f https://raw.githubusercontent.com/google/metallb/v0.7.3/manifests/metallb.yaml

# nginx
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/mandatory.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/baremetal/service-nodeport.yaml

---

入口控制器记录日志：

I0714 22:00:38.056148       7 event.go:258] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"chris-example", Name:"app-ingress", UID:"cbf3b5bf-a67a-11e9-be9a-a4cafa3aa171", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"8681", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress chris-example/app-ingress
I0714 22:01:19.153298       7 event.go:258] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"chris-example", Name:"app-ingress", UID:"cbf3b5bf-a67a-11e9-be9a-a4cafa3aa171", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"8743", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress chris-example/app-ingress
I0714 22:01:38.051694       7 status.go:296] updating Ingress chris-example/app-ingress status from [{192.168.122.253 }] to []
I0714 22:01:38.060044       7 event.go:258] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"chris-example", Name:"app-ingress", UID:"cbf3b5bf-a67a-11e9-be9a-a4cafa3aa171", APIVersion:"networking.k8s.io/v1beta1", ResourceVersion:"8773", FieldPath:""}): type: 'Normal' reason: 'UPDATE' Ingress chris-example/app-ingress

还有 metallb 控制器日志：

{"caller":"main.go:72","event":"noChange","msg":"service converged, no change","service":"kube-system/kube-dns","ts":"2019-07-14T21:58:39.656725017Z"}
{"caller":"main.go:73","event":"endUpdate","msg":"end of service update","service":"kube-system/kube-dns","ts":"2019-07-14T21:58:39.656741267Z"}
{"caller":"main.go:49","event":"startUpdate","msg":"start of service update","service":"chris-example/app-lb","ts":"2019-07-14T21:58:39.6567588Z"}
{"caller":"main.go:72","event":"noChange","msg":"service converged, no change","service":"chris-example/app-lb","ts":"2019-07-14T21:58:39.656842026Z"}
{"caller":"main.go:73","event":"endUpdate","msg":"end of service update","service":"chris-example/app-lb","ts":"2019-07-14T21:58:39.656873586Z"}

---

作为测试，我删除了与 metallb 相关的部署+守护程序集：

kubectl delete deployment -n metallb-system controller
kubectl delete daemonset -n metallb-system speaker

而且设置外网IP后，会再次重置……

Answer 1

我很好奇并重新创建了您的案例。我能够正确地公开服务。

首先：部署自己的 NGINX 时不需要使用 minikube 入口插件。如果这样做，则集群中有 2 个入口控制器，以后会导致混乱。运行：minikube addons disable ingress

旁注：您可以在分配给您的入口的 IP 中看到这种混淆：192.168.122.253，它不在您在configmap-metallb.yaml 中定义的 CIDR 范围 192.168.39.160/28。

---

您需要将ingress-nginx 的服务类型更改为LoadBalancer。你可以通过运行来做到这一点：

kubectl edit -n ingress-nginx service ingress-nginx

此外，您可以将 app-lb 服务更改为 NodePort，因为它不需要暴露在集群之外 - 入口控制器会处理它。

---

说明

从ConfigMap 开始考虑Ingress 对象比Service 更容易。

MetalLB 采用您在ConfigMap 中提供的配置并等待IP 请求API 调用。当它获得一个时，它会从您指定的 CIDR 范围内提供 IP。

以类似的方式，入口控制器（在您的情况下为 NGINX）采用Ingress 对象中描述的配置，并使用它将流量路由到集群中的所需位置。

然后ingress-nginx 服务暴露在集群外部并分配IP。

Ingress 控制器 (NGINX) 根据 Ingress 对象中描述的规则将入站流量定向到应用程序字体中的服务。

图表

Inbound
traffic
   ++     +---------+
   ||     |ConfigMap|
   ||     +--+------+
   ||        |
   ||        | CIDR range to provision
   ||        v
   ||     +--+----------+
   ||     |MetalLB      |               +-------+
   ||     |Load balancer|               |Ingress|
   ||     +-+-----------+               +---+---+
   ||       |                               |
   ||       |  External IP assigned         |Rules described in spec
   ||       |  to service                   |
   ||       v                               v
   ||    +--+--------------------+      +---+------------------+
   ||    |                       |      |  Ingress Controller  |
   |---->+ ingress-nginx service +----->+      (NGINX pod)     |
   +---->|                       +----->+                      |
         +-----------------------+      +----------------------+
                                                    ||
                                                    VV
                                          +-----------------+
                                          | Backend service |
                                          | (app-lb)        |
                                          |                 |
                                          +-----------------+
                                                   ||
                                                   VV
                                          +--------------------+
                                          | Backend pod        |
                                          | (httpbin)          |
                                          |                    |
                                          +--------------------+