【问题标题】：Correctly setting up Istio gateway for GRPC with TLS使用 TLS 为 GRPC 正确设置 Istio 网关
【发布时间】：2020-05-07 14:23:23
【问题描述】：

我一直在尝试设置一个面向外部的 GRPC 支付微服务客户端，使用 tls 自动更新证书。

到目前为止，我已经正确设置了带有证书续订的 certmanager，但是我的网关似乎没有正确转发流量，因为 kubectl -n istio-system describe challenge payments-cert 显示由于 HTTP 404 被返回而导致挑战出错。

我用grpcurl测试了服务的活力，结果是这样的：

Failed to dial target host "payments.mywebsite.com:443": read tcp 192.168.0.16:58849 ->xx.xx.xxx.xx:443: read: connection reset by peer

这说明ip已经被dns正确注册了，而且地址确实是在443上到达的，所以我的Gateway -> VirtualService -> Service -> Deployment设置肯定有问题。

在描述 istio 入口时 (kubectl get svc -n istio-system istio-ingressgateway) 我得到：

istio-ingressgateway LoadBalancer 10.0.13.184 xx.xx.xx.xx ...443:31390/TCP...

这似乎没有将443 转发到预期的端口？

我想让流量进入“payments.mywebsite.com:443”（使用 let-encrypt 自动更新证书）以路由到容器端口 50051 上的我的 grpc 后端。我一直很难破译Istio 文档和资源来完成这项工作！

任何帮助都将受到热烈欢迎！

网关

apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: payments-gateway
  namespace: default
spec:
  selector:
    istio: ingressgateway
  servers:
  - hosts:
    - payments.mywebsite.com
    port:
      name: payments-gateway-https
      number: 443
      protocol: HTTPS
    tls:
      credentialName: payments-cert
      mode: SIMPLE
      privateKey: sds
      serverCertificate: sds

虚拟服务

apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: payments-virtual
spec:
  hosts:
  - "payments.mywebsite.com"
  gateways:
  - payments-gateway
  http:
  - match:
    - uri:
        prefix: /
    route:
    - destination:
        port:
          number: 50051
        host: payments-service

服务

apiVersion: v1
kind: Service
metadata:
  name: payments-service
  labels:
    app: payments-service
spec:
  ports:
  - port: 50051
    name: grpc
  selector:
    app: payments-server

部署

apiVersion: extensions/v1beta1
kind: Deployment
metadata:
  labels:
    app: payments-server
  name: payments-server
spec:
  selector:
    matchLabels:
      app: payments-server
  replicas: 1
  strategy: {}
  template:
    metadata:
      labels:
        app: payments-server
        version: v1
    spec:
      containers:
      - image: gcr.io/mywebsite/payments_server:v1
        name: payments-server
        ports:
        - containerPort: 50051
        resources: {}
      restartPolicy: Always
status: {}

编辑 1：添加来自网关的日志

2020-01-21T14:31:48.841505Z     error   k8s.io/client-go@v11.0.1-0.20190409021438-1a26190bd76a+incompatible/tools/cache/reflector.go:98: Failed to list *v1.Secret: Get https://10.0.0.1:443/api/v1/namespaces/istio-system/secrets?limit=500&resourceVersion=0: net/http: TLS handshake timeout
2020-01-21T14:32:13.173045Z     info    secretFetcherLog        scrtUpdated is called on kubernetes secret payments-cert
2020-01-21T14:32:13.173124Z     warn    secretFetcherLog        failed load server cert/key pair from secret payments-cert: server cert or private key is empty
2020-01-21T14:32:13.173131Z     warn    secretFetcherLog        failed load server cert/key pair from secret payments-cert: server cert or private key is empty
2020-01-21T14:32:13.173137Z     info    secretFetcherLog        secret payments-cert does not change, skip update
2020-01-21T14:32:13.173257Z     info    Trace[1103410]: "Reflector k8s.io/client-go@v11.0.1-0.20190409021438-1a26190bd76a+incompatible/tools/cache/reflector.go:98 ListAndWatch" (started: 2020-01-21 14:31:49.841728972 +0000 UTC m=+44333.359224094) (total time: 23.331269811s):`

编辑 2：添加代理 sidecar 日志（省略承载令牌）

2020-01-21T14:47:51.784617Z     info    curl -k -v -XGET  -H "Accept: application/json, */*" -H "User-Agent: sidecar-injector/v0.0.0 (linux/amd64) kubernetes/$Format" -H "Authorization: Bearer ***" 'https://10.0.0.1:443/apis/admissionregistration.k8s.io/v1beta1/mutatingwebhookconfigurations?fieldSelector=metadata.name%3Distio-sidecar-injector&resourceVersion=299045&timeoutSeconds=449&watch=true'
2020-01-21T14:47:51.787107Z     info    GET https://10.0.0.1:443/apis/admissionregistration.k8s.io/v1beta1/mutatingwebhookconfigurations?fieldSelector=metadata.name%3Distio-sidecar-injector&resourceVersion=299045&timeoutSeconds=449&watch=true 200 OK in 2 milliseconds
2020-01-21T14:47:51.787126Z     info    Response Headers:
2020-01-21T14:47:51.787132Z     info        Audit-Id: 21862701-0185-4e25-b26e-ff78ddb6de1e
2020-01-21T14:47:51.787136Z     info        Content-Type: application/json
2020-01-21T14:47:51.787140Z     info        Date: Tue, 21 Jan 2020 14:47:51 GMT

编辑 3：支付证书结构和集群颁发者

apiVersion: cert-manager.io/v1alpha2
kind: Certificate
metadata:
  name: payments-cert
  namespace: istio-system
spec:
  commonName: payments.mywebsite.com
  dnsNames:
  - payments.mywebsite.com
  issuerRef:
    kind: ClusterIssuer
    name: letsencrypt-prod
  secretName: payments-cert

---

apiVersion: cert-manager.io/v1alpha2
kind: ClusterIssuer
metadata:
 labels:
   name: letsencrypt-prod
 name: letsencrypt-prod
 namespace: istio-system
spec:
 acme:
   email: mywebsite@gmail.com
   solvers:
    - http01:
        ingress:
          class:  istioIngress
   privateKeySecretRef:
     name: letsencrypt-staging
   server: https://acme-v02.api.letsencrypt.org/directory

编辑 4：描述支付证书的输出（+ istio sidecar 注入器的附加日志）

Name:         payments-cert
Namespace:    istio-system
Labels:       <none>
Annotations:  cert-manager.io/certificate-name: payments-cert
              cert-manager.io/issuer-kind: ClusterIssuer
              cert-manager.io/issuer-name: letsencrypt-prod

Type:  kubernetes.io/tls

Data
====
tls.crt:  0 bytes
tls.key:  1679 bytes
ca.crt:   0 bytes

我还查看了谷歌云上 istio sidecar 注入器发出的事件，记录了这些日志：

(Message): No matching pods found   
(Reason): NoPods    
(First Seen): Jan 21, 2020, 2:32:47 PM  
(Last Seen): Jan 21, 2020, 3:28:17 PM   
(counts): 112

【问题讨论】：

从网关 pod 和 Istio 代理 Sidecar 容器以及您的应用程序 pod 发布日志
@ArghyaSadhu 我已经添加了请求的日志，我的应用程序日志表明服务器运行正常
你能把秘密支付证书的内容结构和 istio 版本发布一下吗？
@ArghyaSadhu 添加了请求的 yamls，istio 版本为 1.4.3
你能提供 kubectl describe secret payment-cert -n istio-system 的输出吗

标签： kubernetes google-kubernetes-engine devops istio cert-manager

【解决方案1】：

istio 网关日志failed load server cert/key pair from secret payments-cert: server cert or private key is empty 出错的原因是tls.crt 在秘密payments-cert 中为空。证书管理器生成证书时存在一些问题。

另一件事是您在证书颁发者中混合了letsencrypt-staging和letsencrypt-prod。

这里还有一个example，您可以关注它。

【讨论】：