为什么prometheus服务看不到服务但看到自己？

Question

我有一个 swarm 堆栈。 1 个节点。 Prometheus 服务没有看到服务 whoami。但它确实看到了自己。我也收到来自 whoami 服务的数据，带有“curl 127.0.0.1:800”

我在 Prometheus 中得到的：

docker-stack.yml

    version: '3.7'
    volumes:
        prometheus_data: {}
        grafana_data: {}


services:
  prom:
    image: prom/prometheus
    volumes:
      - ./prometheus/:/etc/prometheus/
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--web.console.libraries=/usr/share/prometheus/console_libraries'
      - '--web.console.templates=/usr/share/prometheus/consoles'
    ports:
      - 8080:9090
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

  grafana:
     image: grafana/grafana
     ports:
       - "3000:3000"
     deploy:
       update_config:
         parallelism: 2
         delay: 10s
       restart_policy:
         condition: on-failure
  whoami:
      image: containous/whoami
      ports:
        - "800:80"
      deploy:
        replicas: 1
        update_config:
          parallelism: 2
          delay: 10s
        restart_policy:
          condition: on-failure
  telegraf:
    image: telegraf
    ports:
      - "998:998"
    deploy:
       replicas: 1
       update_config:
         parallelism: 2
         delay: 10s
       restart_policy:
         condition: on-failure

  alertmanager:
    image: prom/alertmanager
    ports:
      - 9093:9093
    deploy:
      replicas: 1
      update_config:
        parallelism: 2
        delay: 10s
      restart_policy:
        condition: on-failure

prometheus.yml:

# my global config
global:
  scrape_interval:     15s # By default, scrape targets every 15 seconds.
  evaluation_interval: 15s # By default, scrape targets every 15 seconds.
  # scrape_timeout is set to the global default (10s).

  # Attach these labels to any time series or alerts when communicating with
  # external systems (federation, remote storage, Alertmanager).
  external_labels:
      monitor: 'my-project'

# Load and evaluate rules in this file every 'evaluation_interval' seconds.
rule_files:
  - 'alert.rules'
  # - "first.rules"
  # - "second.rules"

# alert
alerting:
  alertmanagers:
  - scheme: http
    static_configs:
    - targets:
      - "alertmanager:9093"

# A scrape configuration containing exactly one endpoint to scrape:
# Here it's Prometheus itself.
scrape_configs:
  # The job name is added as a label `job=<job_name>` to any timeseries scraped from this config.

  - job_name: 'prometheus'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    static_configs:
         - targets: ['127.0.0.1:9090']


  - job_name: 'cadvisor'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    dns_sd_configs:
    - names:
      - 'tasks.cadvisor'
      type: 'A'
      port: 8080

#     static_configs:
#          - targets: ['cadvisor:8080']

  - job_name: 'node-exporter'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    dns_sd_configs:
    - names:
      - 'tasks.node-exporter'
      type: 'A'
      port: 9100

#     static_configs:
#          - targets: ['node-exporter:9100']

  - job_name: 'whoami'

    # Override the global default and scrape targets from this job every 5 seconds.
    scrape_interval: 5s

    dns_sd_configs:
    - names:
      - 'tasks.whoami'
      type: 'A'
      port: 800
    static_configs:
         - targets: ['127.0.0.1:800']

天哪。该网站告诉我应该添加更多文本，因为我的帖子主要是代码并且缺少文本。不知道我还能在请求中添加什么...

Answer 1

Prometheus 在容器内运行，因此 Prometheus 的 127.0.0.1 与您主机的 127.0.0.1 不同。这就是为什么从 Prometheus 调用 127.0.0.1:800/metrics 会导致 Connection Refused。

除此之外，您还将whoami 的端口80 暴露给主机的端口800。所以我认为这些指标最初是从whoami 的端口 80 公开的（对吗？）。如果是这样，这就是来自 Prometheus 的10.0.14.6:800/metrics 拒绝连接的原因。

在 docker-compose 默认网络中，服务可以在不暴露任何端口的情况下相互看到。所以你可以在你的工作目标中使用['whoami:80']。

Answer 2

whoami 服务有问题。我添加了 grafana 服务，舞会显示 grafana 已启动。

也许我在 whoami 的端口上做错了。

更新：

Prometheus 告诉我 strconv.ParseFloat: parsing "f898148383e5": invalid syntax 关于 whoami 服务，而 whoami 服务自己告诉我“f898148383e5”是 ?container 的主机名