【发布时间】：2021-01-18 23:41:05
【问题描述】：
我正在使用 Istio 1.8.0 和本地 k8s v1.19..我们有几个微服务正在运行，我使用 STRICT 模式进行对等身份验证。而且我可以验证，如果我使用PERMISSIVE 模式，我没有收到任何 503 错误。
我真的很难找到任何解决方案，因为我不想按照建议使用PERMISSIVE 模式。
这是 istio ingressgateway 的日志。
$kubectl logs -f istio-ingressgateway-75496c97df-44g6l -n istio-system
[2021-01-16T07:28:51.852Z] "GET /config HTTP/1.1" 503 URX "-" 0 95 57 57 "95.0.145.40,10.6.0.21" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "foo-bar.example.com" "10.6.25.34:3000" outbound|80||oneapihub-ui-dev.hub-dev.svc.cluster.local 10.6.5.216:46364 10.6.5.216:8080 10.6.0.21:14387 - -
这是我的 ui 微服务的日志。我没有发现其他服务的任何 503 错误
$ kubectl get ns --show-labels
NAME STATUS AGE LABELS
hub-dev Active 2d19h istio-injection=enabled
istio-system Active 3d5h istio-injection=disabled
$ kubectl get pods -n hub-dev
NAME READY STATUS RESTARTS AGE
oneapihub-api-dev-79dff67cdb-hx754 3/3 Running 0 15h
oneapihub-auth-dev-76cfcb6cb4-74ljq 3/3 Running 0 15h
oneapihub-backend-dev-d76799bcd-bmwjn 2/2 Running 0 15h
oneapihub-cronjob-dev-6879dbf9b8-wvpnp 3/3 Running 0 15h
oneapihub-mp-dev-864794d446-cfqj7 3/3 Running 0 15h
oneapihub-ui-dev-67d7bb6779-8z4xt 2/2 Running 0 14h
redis-hub-master-0 2/2 Running 0 15h
$ kubectl logs -f oneapihub-ui-dev-67d7bb6779-8z4xt -n hub-dev -c istio-proxy
[2021-01-15T14:17:24.698Z] "GET /config HTTP/1.1" 503 URX "-" 0 95 65 64 "95.0.145.40,10.6.0.19" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/87.0.4280.141 Safari/537.36" "foo-bar.example.com" "10.6.25.241:3000" outbound|80||oneapihub-ui-dev.hub-dev.svc.cluster.local 10.6.5.216:37584 10.6.5.216:8080 10.6.0.19:31138 - -
[2021-01-15T14:17:24.817Z]
这里是对等身份验证
$ kubectl get peerauthentication -n istio-system -o yaml
apiVersion: v1
items:
- apiVersion: security.istio.io/v1beta1
kind: PeerAuthentication
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"security.istio.io/v1beta1","kind":"PeerAuthentication","metadata":{"annotations":{},"name":"default","namespace":"istio-system"},"spec":{"mtls":{"mode":"STRICT"}}}
name: default
namespace: istio-system
spec:
mtls:
mode: STRICT
这是我的前端应用的 service.yaml。
capel0068340585:~ semural$ kubectl get svc -n hub-dev
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
oneapihub-ui-dev ClusterIP 10.254.47.95 <none> 80/TCP 48m
# Source: oneapihub-ui/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
name: RELEASE-NAME-oneapihub-ui
labels:
app.kubernetes.io/name: oneapihub-ui
app.kubernetes.io/instance: RELEASE-NAME
app.kubernetes.io/managed-by: Helm
helm.sh/chart: oneapihub-ui-0.1.0
spec:
type: ClusterIP
ports:
- port: 80
targetPort: 3000
protocol: TCP
name: http
这是我在运行微服务的同一个命名空间中创建的网关和 VS。
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
name: hub-gateway
spec:
selector:
istio: ingressgateway
servers:
- port:
number: 80
name: http
protocol: HTTP
hosts:
- foo-bar.example.com
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
name: hub
spec:
hosts:
- foo-bar.example.com
gateways:
- hub-gateway
http:
- route:
- destination:
host: oneapihub-ui-dev.hub-dev.svc.cluster.local
port:
number: 80
这是 ingressgateway 的代理配置
$ istioctl proxy-config route istio-ingressgateway-75496c97df-44g6l -n istio-system -o json
[
{
"name": "http.80",
"virtualHosts": [
{
"name": "foo-baexample.com:80",
"domains": [
"foo-bar.example.com",
"foo-bar.example.com:*"
],
"routes": [
{
"match": {
"prefix": "/"
},
"route": {
"cluster": "outbound|80||oneapihub-ui-dev.hub-dev.svc.cluster.local",
"timeout": "0s",
"retryPolicy": {
"retryOn": "connect-failure,refused-stream,unavailable,cancelled,retriable-status-codes",
"numRetries": 2,
"retryHostPredicate": [
{
"name": "envoy.retry_host_predicates.previous_hosts"
}
],
这是 istioctl describe pods 的输出。
$ istioctl x describe pod oneapihub-api-dev-78fbccf48c-5hb4c -n hub-dev
Pod: oneapihub-api-dev-78fbccf48c-5hb4c
Pod Ports: 50004 (oneapihub-api), 15090 (istio-proxy)
--------------------
Service: oneapihub-api-dev
Port: http 80/HTTP targets pod port 50004
$ istioctl x describe pod oneapihub-auth-dev-7f8998cd69-gzmnm -n hub-dev
Pod: oneapihub-auth-dev-7f8998cd69-gzmnm
Pod Ports: 50002 (oneapihub-auth), 15090 (istio-proxy)
--------------------
Service: oneapihub-auth-dev
Port: http 80/HTTP targets pod port 5000
$ istioctl x describe pod oneapihub-backend-dev-849b4bcd5d-fcm4l -n hub-dev
Pod: oneapihub-backend-dev-849b4bcd5d-fcm4l
Pod Ports: 50001 (oneapihub-backend), 15090 (istio-proxy)
--------------------
Service: oneapihub-backend-dev
Port: http 80/HTTP targets pod port 50001
$ istioctl x describe pod oneapihub-cronjob-dev-58b64d9c68-lv5bk -n hub-dev
Pod: oneapihub-cronjob-dev-58b64d9c68-lv5bk
Pod Ports: 50005 (oneapihub-cronjob), 15090 (istio-proxy)
--------------------
Service: oneapihub-cronjob-dev
Port: http 80/HTTP targets pod port 50005
$ istioctl x describe pod oneapihub-mp-dev-74fd6ffc9f-65gh5 -n hub-dev
Pod: oneapihub-mp-dev-74fd6ffc9f-65gh5
Pod Ports: 50003 (oneapihub-mp), 15090 (istio-proxy)
--------------------
Service: oneapihub-mp-dev
Port: http 80/HTTP targets pod port 50003
$ istioctl x describe pod oneapihub-ui-dev-7fd56f747c-nr5fk -n hub-dev
Pod: oneapihub-ui-dev-7fd56f747c-nr5fk
Pod Ports: 3000 (oneapihub-ui), 15090 (istio-proxy)
--------------------
Service: oneapihub-ui-dev
Port: http 80/HTTP targets pod port 3000
Exposed on Ingress Gateway http://53.6.48.168
VirtualService: hub
1 HTTP route(s)
$ istioctl x describe pod redis-hub-master-0 -n hub-dev
Pod: redis-hub-master-0
Pod Ports: 6379 (redis-hub), 15090 (istio-proxy)
Suggestion: add 'version' label to pod for Istio telemetry.
--------------------
Service: redis-hub-headless
Port: redis 6379/Redis targets pod port 6379
--------------------
Service: redis-hub-master
Port: redis 6379/Redis targets pod port 6379
这是我的目的地规则
apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
name: backend-destination-rule
namespace: hub-dev
spec:
host: oneapihub-backend-dev.hub-dev.svc.cluster.local
trafficPolicy:
tls:
mode: ISTIO_MUTUAL
sni: oneapihub-backend-dev.hub-dev.svc.cluster.local
【问题讨论】：
-
你有目的地规则吗？否则，您需要在目标规则中添加 trafficPolicy: tls: mode: ISTIO_MUTUAL 。这是 503 错误原因istio.io/latest/docs/ops/common-problems/network-issues/…
-
@Shay 是的，我添加如下但仍然相同.. /config 似乎有些问题但我仍然无法解决问题
tls: mode: ISTIO_MUTUAL sni: oneapihub-backend-dev.hub-dev.svc.cluster.local -
您是否介意将目标规则添加到问题中，否则我建议仅将 PeerAuthentication 应用于 hub-dev 命名空间，而不是整个集群。然后你可以做一些事情 mtls: mode: STRICT portLevelMtls: 80: mode: DISABLE
-
@Shay，我已经添加了.. Current PeerAuthentication apply only hub-dev namespace 我也试过
-
您找到解决方案了吗？谈到禁用 pod 在不同 ns 之间进行通信，您必须定义网络策略 - 阅读更多：istio.io/latest/docs/ops/common-problems/injection、stackoverflow.com/questions/43675572/…。
标签： kubernetes istio