为 confluent cloud kafka 等外部服务配置 prometheus 服务监视器答案

【问题标题】：configure prometheus service monitor for external services like confluent cloud kafka为 confluent cloud kafka 等外部服务配置 prometheus 服务监视器
【发布时间】：2022-08-16 16:40:55
【问题描述】：

我正在尝试配置 prometheus 服务监视器以获取融合云（kafka）指标。根据他们的文档 (https://api.telemetry.confluent.cloud/docs?&_ga=2.215665047.1225578079.1655110920-914939436.1654166692#tag/Version-2/paths/~1v2~1metrics~1{dataset}~1export/get)，他们已经使用 scrape_config 进行了配置。

但我想使用服务监视器进行抓取。

我创建了一个 k8s 外部服务：

kind: Service
apiVersion: v1
metadata:
  name: kafka-metric-api-service
  namespace: monitoring
  labels:
    app: kafka-metric-api
spec:
  type: ExternalName
  externalName: api.telemetry.confluent.cloud

我创建了如下服务监视器：

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: kafka-metric-api
  namespace: monitoring
  labels:
    app: kafka-metric-api
spec:
  endpoints: 
  - basicAuth:
      password:
        name: kafka-basic-auth
        key: password
      username:
        name: kafka-basic-auth
        key: username
    path: /v2/metrics/cloud/export
    interval: 60s
    scheme: https
    tlsConfig:
      insecureSkipVerify: true
    params:
      resource.kafka.id:
        - lkc-xxxx
  selector:
    matchLabels:
      app: kafka-metric-api

由于服务没有任何端口，端口名称，我不知道在spec.endpoint中放置什么。当前设置不起作用，我看到以下状态：

有人可以帮助我了解如何为没有端口的 confluent kafka 等外部服务配置服务监视器以及如何抓取指标吗？在互联网上，我看到了定义了端口的服务示例。

标签： prometheus kubernetes-service prometheus-operator confluent-cloud servicemonitor

【解决方案1】：

这对我有用

apiVersion: v1
kind: Endpoints
metadata:
  name: confluent-cloud-telemetry-svc
  namespace: monitoring
  labels:
    app: confluent-cloud-telemetry
    release: prometheus-operator
subsets:
- addresses:
  - ip: 54.149.69.190
  ports:
  - name: confluent-cloud-telemetry-port
    protocol: TCP
    port: 443

---
apiVersion: v1
kind: Service
metadata:
  name: confluent-cloud-telemetry-svc
  namespace: monitoring
  labels:
    app: confluent-cloud-telemetry
    release: prometheus-operator
spec:
  type: ExternalName
  externalName: api.telemetry.confluent.cloud
  ports:
  - name: confluent-cloud-telemetry-port
    protocol: TCP
    port: 443
    targetPort: 443

---
apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  name: confluent-cloud-telemetry-sm
  namespace: monitoring
  labels:
    app: confluent-cloud-telemetry
    release: prometheus-operator
spec:
  selector:
    matchLabels:
      app: confluent-cloud-telemetry
      release: prometheus-operator
  namespaceSelector:
    matchNames: 
    - monitoring
  endpoints: 
  - basicAuth:
      password:
        name: kafka-basic-auth
        key: password
      username:
        name: kafka-basic-auth
        key: user
    port: confluent-cloud-telemetry-port
    interval: 60s
    honorLabels: true
    scheme: https
    path: /v2/metrics/cloud/export
    tlsConfig:
      insecureSkipVerify: true
    relabelings:
      - sourceLabels: [__address__]
        targetLabel: __address__
        regex: (.*)
        replacement: "api.telemetry.confluent.cloud:443"
        action: replace
    params:
      resource.kafka.id:
        - <YOUR_CLUSTER_ID>

---
apiVersion: v1
kind: Secret
metadata:
  name: kafka-basic-auth
data:
  password: <YOUR_SECRET_BASE64>
  user: <YOUR_KEY_BASE64>
type: Opaque

【讨论】：