【发布时间】：2022-10-13 05:33:04
【问题描述】：
在使用 ingress-nginx 时，我一直在查看很多关于 503 错误的帖子，但没有找到适用于我的设置的解决方案。
在本地 Oracle Linux Server v8.5 上运行。使用 k3s，以及节点的以下信息：
kubectl get node -o wide
NAME STATUS ROLES AGE VERSION INTERNAL-IP EXTERNAL-IP OS-IMAGE KERNEL-VERSION CONTAINER-RUNTIME
mptdevws00 Ready master 6d21h v1.19.2+k3s1 192.168.10.100 <none> Oracle Linux Server 8.5 5.4.17-2102.201.3.el8uek.x86_64 containerd://1.4.0-k3s1
运行位于https://raw.githubusercontent.com/kubernetes/ingress-nginx/controller-v1.4.0/deploy/static/provider/baremetal/deploy.yaml 的裸机 ingress-nginx
我正在部署一个非常基本的 hello-word 部署、服务和入口，定义如下：
---
apiVersion: v1
kind: Service
metadata:
name: hello-world-service
spec:
type: NodePort
ports:
- port: 80
selector:
app: hello-world
---
apiVersion: apps/v1
kind: Deployment
metadata:
name: hello-world-test
spec:
selector:
matchLabels:
app: hello-world
replicas: 1
template:
metadata:
labels:
app: hello-world
spec:
containers:
- name: nginx
image: nginx
ports:
- containerPort: 80
volumeMounts:
- name: hello-world-volume
mountPath: /usr/share/nginx/html
volumes:
- name: hello-world-volume
configMap:
name: hello-world
---
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
name: hello-world-ingress
annotations:
nginx.ingress.kubernetes.io/ssl-redirect: "false"
spec:
rules:
- http:
paths:
- path: /hello
pathType: Prefix
backend:
service:
name: hello-world-service
port:
number: 80
不得不运行 mod_probe ip_tables 来加载它，而且我每次启动 ingress-nginx 部署时都必须运行 kubectl delete -A ValidatingWebhookConfiguration ingress-nginx-admission 因为验证 webhook 不断吐出一个 Unknown Host 错误，所以我刚转身直到我得到这个工作为止。
Pod、服务、入口和端点似乎可以正常启动：
kubectl get pods --all-namespaces -o wide
NAMESPACE NAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATES
kube-system metrics-server-7b4f8b595-kp4j4 1/1 Running 0 6d21h 10.42.0.2 mptdevws00 <none> <none>
kube-system local-path-provisioner-7ff9579c6-s6f4z 1/1 Running 1 6d21h 10.42.0.4 mptdevws00 <none> <none>
kube-system coredns-66c464876b-4g5q6 1/1 Running 0 6d21h 10.42.0.3 mptdevws00 <none> <none>
ingress-nginx svclb-ingress-nginx-controller-hc8gb 2/2 Running 0 58m 10.42.0.206 mptdevws00 <none> <none>
ingress-nginx ingress-nginx-admission-create-m9nfw 0/1 Completed 0 58m 10.42.0.207 mptdevws00 <none> <none>
ingress-nginx ingress-nginx-admission-patch-kdvdf 0/1 Completed 0 58m 10.42.0.208 mptdevws00 <none> <none>
ingress-nginx ingress-nginx-controller-5979dd67b-lrg5r 1/1 Running 0 58m 10.42.0.209 mptdevws00 <none> <none>
default hello-world-test-59cc4bc76f-b58k9 1/1 Running 0 57m 10.42.0.212 mptdevws00 <none> <none>
kubectl get service --all-namespaces -o wide
NAMESPACE NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE SELECTOR
default kubernetes ClusterIP 10.43.0.1 <none> 443/TCP 6d21h <none>
kube-system kube-dns ClusterIP 10.43.0.10 <none> 53/UDP,53/TCP,9153/TCP 6d21h k8s-app=kube-dns
kube-system metrics-server ClusterIP 10.43.191.252 <none> 443/TCP 6d21h k8s-app=metrics-server
ingress-nginx ingress-nginx-controller-admission ClusterIP 10.43.141.95 <none> 443/TCP 59m app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
ingress-nginx ingress-nginx-controller LoadBalancer 10.43.191.206 192.168.10.100 80:32700/TCP,443:31849/TCP 59m app.kubernetes.io/component=controller,app.kubernetes.io/instance=ingress-nginx,app.kubernetes.io/name=ingress-nginx
default hello-world-service NodePort 10.43.56.86 <none> 80:31043/TCP 58m app=hello-world
kubectl get ingress --all-namespaces -o wide
Warning: extensions/v1beta1 Ingress is deprecated in v1.14+, unavailable in v1.22+; use networking.k8s.io/v1 Ingress
NAMESPACE NAME CLASS HOSTS ADDRESS PORTS AGE
default hello-world-ingress nginx * 192.168.10.100 80 49m
kubectl get ep --all-namespaces -o wide
NAMESPACE NAME ENDPOINTS AGE
default kubernetes 192.168.10.100:6443 6d21h
kube-system metrics-server 10.42.0.2:443 6d21h
kube-system kube-dns 10.42.0.3:53,10.42.0.3:9153,10.42.0.3:53 6d21h
ingress-nginx ingress-nginx-controller 10.42.0.209:443,10.42.0.209:80 60m
ingress-nginx ingress-nginx-controller-admission 10.42.0.209:8443 60m
default hello-world-service 10.42.0.212:80 59m
kube-system rancher.io-local-path <none> 6d21h
如果我做一个kubectl exec -it --namespace=ingress-nginx ingress-nginx-controller-5979dd67b-lrg5r -- curl http://hello-world-service.default.svc/它返回基本的 Hello World html 很好，所以我知道服务和部署至少可以正常工作。
如果我 curl http://localhost/ （或使用主机 ip 或主机名）我会得到一个 404，这是我基于 nginx.conf 对 / 的响应所期望的。所以这告诉我入口点至少已经启动并正在运行。 http://localhost/healthz 返回一个 200，我希望再次基于 nginx.conf
但是，执行 curl http://localhost/hello 总是返回 503。
在控制器日志中，我刚刚得到：
I1011 20:34:13.274379 8 controller.go:168] "Configuration changes detected, backend reload required"
I1011 20:34:13.406122 8 controller.go:185] "Backend successfully reloaded"
I1011 20:34:13.407450 8 event.go:285] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-5979dd67b-lrg5r", UID:"49499847-4f0f-4196-ba71-81980445b46c", APIVersion:"v1", ResourceVersion:"444384", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
W1011 20:34:21.685551 8 reflector.go:424] k8s.io/client-go@v0.25.2/tools/cache/reflector.go:169: failed to list *v1.EndpointSlice: the server could not find the requested resource
E1011 20:34:21.686258 8 reflector.go:140] k8s.io/client-go@v0.25.2/tools/cache/reflector.go:169: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: the server could not find the requested resource
I1011 20:34:30.136621 8 store.go:430] "Found valid IngressClass" ingress="default/hello-world-ingress" ingressclass="nginx"
W1011 20:34:30.137527 8 endpointslices.go:81] Error obtaining Endpoints for Service "default/hello-world-service": no object matching key "default/hello-world-service" in local store
W1011 20:34:30.137569 8 controller.go:1112] Service "default/hello-world-service" does not have any active Endpoint.
I1011 20:34:30.137664 8 controller.go:168] "Configuration changes detected, backend reload required"
I1011 20:34:30.142794 8 event.go:285] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"hello-world-ingress", UID:"3d297335-e2eb-46cf-a6a5-56bf68725c8f", APIVersion:"networking.k8s.io/v1", ResourceVersion:"447873", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
I1011 20:34:30.250894 8 controller.go:185] "Backend successfully reloaded"
I1011 20:34:30.251447 8 event.go:285] Event(v1.ObjectReference{Kind:"Pod", Namespace:"ingress-nginx", Name:"ingress-nginx-controller-5979dd67b-lrg5r", UID:"49499847-4f0f-4196-ba71-81980445b46c", APIVersion:"v1", ResourceVersion:"444384", FieldPath:""}): type: 'Normal' reason: 'RELOAD' NGINX reload triggered due to a change in configuration
I1011 20:34:58.695644 8 status.go:299] "updating Ingress status" namespace="default" ingress="hello-world-ingress" currentValue=[] newValue=[{IP:192.168.10.100 Hostname: Ports:[]}]
W1011 20:34:58.702687 8 endpointslices.go:81] Error obtaining Endpoints for Service "default/hello-world-service": no object matching key "default/hello-world-service" in local store
W1011 20:34:58.702735 8 controller.go:1112] Service "default/hello-world-service" does not have any active Endpoint.
I1011 20:34:58.703331 8 event.go:285] Event(v1.ObjectReference{Kind:"Ingress", Namespace:"default", Name:"hello-world-ingress", UID:"3d297335-e2eb-46cf-a6a5-56bf68725c8f", APIVersion:"networking.k8s.io/v1", ResourceVersion:"447926", FieldPath:""}): type: 'Normal' reason: 'Sync' Scheduled for sync
W1011 20:34:59.764277 8 reflector.go:424] k8s.io/client-go@v0.25.2/tools/cache/reflector.go:169: failed to list *v1.EndpointSlice: the server could not find the requested resource
E1011 20:34:59.764372 8 reflector.go:140] k8s.io/client-go@v0.25.2/tools/cache/reflector.go:169: Failed to watch *v1.EndpointSlice: failed to list *v1.EndpointSlice: the server could not find the requested resource
10.42.0.206 - - [11/Oct/2022:20:35:19 +0000] "GET /hello HTTP/1.1" 503 190 "-" "curl/7.61.1" 78 0.000 [default-hello-world-service-80] [] - - - - 3cc8c307cea5d96642b37e10a01707f1
如果我添加 /、/%2 等的重写目标都没关系。我已经尝试为我的服务定义一个类型：NodePort，或者将其关闭。不会改变任何东西。似乎无论人们遇到什么其他问题，我似乎都无法解决这个问题。
我还尝试在各种容器中执行以命中服务的 ips、服务名称等等，这看起来很好，这只是入口点和服务失败之间的路径。在我的 ingress-nginx 部署中打开调试并没有提供任何指向问题的细节（至少我可以理解）。
非常感谢任何可能指导我找到解决方案的帮助，或者至少帮助追踪发生破损的位置。当我从 pod 中运行 curl 时，实际的 hello-world pod 中的日志显示成功的请求，但是当我得到 503 时没有显示任何内容，因此请求甚至没有命中服务。
【问题讨论】：
标签： k3s ingress-nginx