Kubernetes 使用一个代码库但配置不同（环境变量）的多个部署答案

【问题标题】：kubernetess multiple deployments using one code base but different configuration (environement variables)Kubernetes 使用一个代码库但配置不同（环境变量）的多个部署
【发布时间】：2021-07-12 20:04:34
【问题描述】：

我有一个项目，我们正在使用来自 kafka 的数据并发布到 mongo。事实上，代码库只完成一项任务，可能是 mongo 到 kafka 的迁移、kafka 到 mongo 的迁移或其他。

我们必须从不同的 kafka 主题中消费并发布到不同的 mongo 集合。现在这些是并行的工作流。

当前的设计是拥有一个可以从任何主题消费并发布到任何 mongo 集合的代码库，该集合可以使用环境变量进行配置。所以我们创建了一个 Kubernetes Pod，里面有多个容器。每个容器都有不同的环境变量。

我的问题：

在一个 pod 中使用多个容器是否明智。很容易区分，但由于它们是紧密耦合的，我猜测失败的可能性很大，实际上微服务设计并不正确。
是否应该为每个管道创建多个部署？维护起来会非常困难，因为每个都有不同的部署配置。
有没有更好的方法来解决这个问题？

第 1 步示例：

apiVersion: apps/v1
kind: Deployment
metadata:
  annotations: {}
  name: test-raw-mongodb-sink-apps
  namespace: test-apps
spec:
  selector:
    matchLabels:
      app: test-raw-mongodb-sink-apps
  template:
    metadata:
      labels:
        app: test-raw-mongodb-sink-apps
    spec:
      containers:
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-alchemy
        - name: INPUT_TOPIC
          value: test.raw.ptv.alchemy
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8081"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/dpl/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-alchemy
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-bloomberg
        - name: INPUT_TOPIC
          value: test.raw.pretrade.bloomberg
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8082"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-bloomberg
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-calypso
        - name: INPUT_TOPIC
          value: test.raw.ptv.calypso
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8083"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-calypso
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-dtres
        - name: INPUT_TOPIC
          value: test.raw.ptv.dtres
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8084"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-dtres
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-feds
        - name: INPUT_TOPIC
          value: test.raw.ptv.feds
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8085"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-feds
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-hoops
        - name: INPUT_TOPIC
          value: test.raw.ptv.hoops
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8086"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-hoops
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxcore
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_core
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8087"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxcore
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxeqd
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_eqd_sa
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8088"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxeqd
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxgts
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_gts_sa
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8089"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxgts
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxmr
        - name: INPUT_TOPIC
          value: test.raw.ptv.murex_mr
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8090"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxmr
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxgtscf
        - name: INPUT_TOPIC
          value: test.raw.cashflow.murex_gts_sa
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8091"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxgtscf
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxcoll
        - name: INPUT_TOPIC
          value: test.raw.collateral.mxcoll
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8092"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxcoll
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-mxcoll-link
        - name: INPUT_TOPIC
          value: test.raw.collateral.mxcoll_link
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8093"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-mxcoll-link
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-ost
        - name: INPUT_TOPIC
          value: test.raw.ptv.ost
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8094"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-ost
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true
      - env:
        - name: EVENTS_TOPIC
          value: test.ops.proc-events
        - name: GROUP_ID
          value: test-mongodb-sink-posmon
        - name: INPUT_TOPIC
          value: test.raw.ptp.posmon
        - name: MONGODB_AUTH_DB
          value: admin
        - name: MONGODB_HOST0
          value: test-mongodb-0.test-mongodb-headless.test-infra
        - name: MONGODB_HOST1
          value: test-mongodb-1.test-mongodb-headless.test-infra
        - name: MONGODB_PASSWORD
          value: test123
        - name: MONGODB_PORT
          value: "27017"
        - name: MONGODB_USERNAME
          value: root
        - name: SERVER_PORT
          value: "8095"
        - name: KAFKA_BROKERS
          value: kafka-cluster-kafka-bootstrap.kafka:9093
        - name: TRUSTSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: ca.password
              name: kafka-ca-cert
        - name: KEYSTORE_PASSWORD
          valueFrom:
            secretKeyRef:
              key: user.password
              name: kafka
        image: tools.testCompany.co.za:8093/local/tt--mongodb-map:0.0.7.0-SNAPSHOT
        name: test-mongodb-sink-posmon
        securityContext:
          allowPrivilegeEscalation: true
          privileged: true
        volumeMounts:
        - mountPath: /app/resources
          name: properties
        - mountPath: /stores
          name: stores
          readOnly: true

谢谢

【问题讨论】：

标签： kubernetes kubernetes-pod kubernetes-deployment

【解决方案1】：

Helm 之类的模板工具可让您从部署时设置中填写环境变量值。在 Helm 中，这看起来像：

env:
  - name: EVENTS_TOPIC
    value: {{ .Values.eventsTopic }}
  - name: GROUP_ID
    value: {{ .Values.groupId }}
  - name: INPUT_TOPIC
    value: {{ .Values.inputTopic }}

然后您可以使用不同的主题集多次部署它：

helm install alchemy . \
  --set eventsTopic=test.ops.proc-events \
  --set groupId=test-mongodb-sink-alchemy \
  --set inputTopic=test.raw.ptv.alchemy
helm install bloomberg . \
  --set eventsTopic=test.ops.proc-events \
  --set groupId=test-mongodb-sink-bloomberg \
  --set inputTopic=test.raw.pretrade.bloomberg

您也可以编写 Helm 图表以配置主题集列表，并且只部署一次：

{{- $top := . -}}{{-/* because "range" overwrites "." */-}}
{{- $topic := range $topics -}}
---
apiVersion: v1
kind: Deployment
metadata:
  name: {{ $topic.name }}
spec:
  ...
    env:
      - name: EVENT_TOPIC
        value: {{ $top.Values.eventTopic }}{{/* common to all deployments */}}
      - name: GROUP_ID
        value: test-mongodb-sink-{{ $topic.name }}
      - name: INPUT_TOPIC
        value: {{ $topic.inputTopic }}

编写如下配置：

eventTopic: test.ops.proc-events
topics:
  - name: alchemy
    inputTopic: test.raw.ptv.alchemy
  - name: bloomberg
    inputTopic: test.raw.pretrade.bloomberg

然后像这样部署：

helm install connector . -f topic-listing.yaml

无论如何，每个 pod 只需要一个容器。这有几个原因。如果主题列表发生变化，这使您可以创建或删除部署，而不会干扰其他主题；如果所有内容都在一个 pod 中，则您必须停止并一起重新启动所有内容，卡夫卡可能需要一两分钟才能弄清楚发生了什么。在 Kafka 上下文中，您还可以运行与主题上的分区一样多的消费者，但不会更多；如果您有一个非常繁忙的主题，您可以轻松地将该部署的 replicas: 设置为多个分区的多个使用者，但如果所有内容都在一个 pod 中，您唯一的选择是将所有内容一起扩展。

【讨论】：

是的，每个 pod 一个容器听起来很理想。但想象一下 pod 的数量等于 kafka 主题的数量。这将使 kubernetes 的前端完全无用。你觉得呢？
如果你有足够多的主题来管理它们是一个问题：还要考虑到 pod 中的每个容器都保证在同一个节点上运行，因此将所有内容打包到一个 pod 中可能会压倒节点它继续运行。每个 pod 一个容器可以让事情更好地分布在集群中。

【解决方案2】：

在一个 pod 中使用多个容器是否明智。很容易区分，但由于它们是紧密耦合的，我猜测失败的可能性很高，实际上微服务设计并不正确。

您很可能希望将它们部署为单独的服务，以便您可以独立地更新或重新配置它们。

我应该为每个管道创建多个部署吗？维护起来会非常困难，因为每个都有不同的部署配置。

Kustomize 是 kubectl 中的内置工具，当您想在具有不同配置的多个环境中部署相同的清单时，它是一个不错的选择。除了kubectl，此解决方案不需要其他工具。

使用 Kustomize 部署到多个环境

目录结构：

base/
  - deployment.yaml      # fully deployable manifest - no templating
  - kustomization.yaml   # default values e.g. for dev environment
app1/
  - kustomization.yaml   # specific values for app1
app2/
  - kustomization.yaml   # specific values for app2

使用 Kustomization 的示例部署清单

这里，环境变量是从ConfigMap 加载的，这样我们就可以使用configMapGenerator。这个文件是base/deployment.yaml

apiVersion: apps/v1
kind: Deployment
metadata:
  name: mongodb-sink
  namespace: test-apps
spec:
  template: // some fiels, e.g. labels are omitted in example
    spec:
      containers:
      - name: mongodb-sink
        image: mongodb-map:0.0.7.0-SNAPSHOT
        env:
          - name: MONGODB_HOST0
            value: test-mongodb-0.test-mongodb-headless.test-infra
          - name: MONGODB_HOST1
            value: test-mongodb-1.test-mongodb-headless.test-infra
          - name: GROUP_ID
            valueFrom:
              configMapKeyRef:
                name: my-values
                key: GROUP_ID
          - name: INPUT_TOPIC
            valueFrom:
              configMapKeyRef:
                name: my-values
                key: INPUT_TOPIC
      ...

另外添加一个base/kustomization.yaml文件来描述configMapGenerator和相关文件。

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

resources:
- deployment.yaml

configMapGenerator:
- name: my-values
  behavior: replace
  literals:
  - GROUP_ID=test-mongodb-sink-calypso
  - INPUT_TOPIC=test.raw.ptv.calypso
  ... # also add your other values

预览清单

kubectl kustomize base/

应用清单

kubectl apply -k base/

为 app1 和 app2 添加配置

有了 app1，我们现在想使用我们在 base/ 中的清单，并覆盖 app1 的不同之处。这个文件是app1/kustomization.yaml 和app2/kustomization.yaml 类似。

apiVersion: kustomize.config.k8s.io/v1beta1
kind: Kustomization

bases:
- ../base

namePrefix: bloomberg-sink-   # this gives your Deployment a prefixed name

configMapGenerator:
- name: my-values
  behavior: replace
  literals:
  - GROUP_ID=test-mongodb-sink-bloomberg
  - INPUT_TOPIC=test.raw.pretrade.bloomberg
  ... # also add your other values

预览清单

kubectl kustomize app1/

应用清单

kubectl apply -k app1/

文档

【讨论】：