使用 docker-compose 在端口 9100 上无法访问 Prometheus Nodeexporter

Question

我使用 Docker Compose（docker-compose.yml 如下所示）在同一个 Ubuntu 18.04 系统上启动了 Prometheus、Grafana 和 Nodeexporter。

问题：我无法通过http://<ip.address>:9100/metrics 访问Nodeexporter，而Promethus 和Grafana 可以分别通过http://<ip.address>:9090 和http://<ip.address>:3000 访问。

nodeexporter 的 docker 日志显示它正在侦听端口 9100，但 lsof 没有显示这一点。

为什么nodeexporter不能监听9100端口？

docker logs nodeexporter的输出

time="2020-01-21T16:38:07Z" level=info msg="Starting node_exporter (version=0.18.1, branch=HEAD, revision=3db77732e925c08f675d7404a8c46466b2ece83e)" source="node_exporter.go:156"
time="2020-01-21T16:38:07Z" level=info msg="Build context (go=go1.12.5, user=root@b50852a1acba, date=20190604-16:41:18)" source="node_exporter.go:157"
time="2020-01-21T16:38:07Z" level=info msg="Enabled collectors:" source="node_exporter.go:97"
time="2020-01-21T16:38:07Z" level=info msg=" - arp" source="node_exporter.go:104"
...
time="2020-01-21T16:38:07Z" level=info msg=" - xfs" source="node_exporter.go:104"
time="2020-01-21T16:38:07Z" level=info msg=" - zfs" source="node_exporter.go:104"
time="2020-01-21T16:38:07Z" level=info msg="Listening on :9100" source="node_exporter.go:170"

lsof -i :3000,9100,9090的输出

COMMAND     PID USER   FD   TYPE  DEVICE SIZE/OFF NODE NAME
docker-pr 22163 root    4u  IPv6 3751831      0t0  TCP *:9090 (LISTEN)
docker-pr 22202 root    4u  IPv6 3751884      0t0  TCP *:3000 (LISTEN)

curl localhost:9100的输出

curl: (7) Failed to connect to localhost port 9100: Connection refused

docker-compose.yml

改编自https://github.com/stefanprodan/dockprom/blob/master/docker-compose.yml

version: '2.1'

networks:
  monitor-net:
    driver: bridge
  default:
      external:
        name: ${NETWORK}

services:

  prometheus:
    image: prom/prometheus:v2.15.2
    container_name: prometheus
    volumes:
      - ./prometheus:/etc/prometheus
      - prometheus_data:/prometheus
    command:
      - '--config.file=/etc/prometheus/prometheus.yml'
      - '--storage.tsdb.path=/prometheus'
      - '--web.console.libraries=/etc/prometheus/console_libraries'
      - '--web.console.templates=/etc/prometheus/consoles'
      - '--storage.tsdb.retention.time=200h'
      - '--web.enable-lifecycle'
    restart: unless-stopped
    expose:
      - 9090
    networks:
      - monitor-net
    labels:
      org.label-schema.group: "monitoring"

  nodeexporter:
    image: prom/node-exporter:v0.18.1
    container_name: nodeexporter
    volumes:
      - /proc:/host/proc:ro
      - /sys:/host/sys:ro
      - /:/rootfs:ro
    command:
      - '--path.procfs=/host/proc'
      - '--path.rootfs=/rootfs'
      - '--path.sysfs=/host/sys'
      - '--collector.filesystem.ignored-mount-points=^/(sys|proc|dev|host|etc)($$|/)'
    restart: unless-stopped
    expose:
      - 9100
    networks:
      - monitor-net
    labels:
      org.label-schema.group: "monitoring"


  grafana:
    image: grafana/grafana:6.5.3
    container_name: grafana
    volumes:
      - grafana_data:/var/lib/grafana
      - ./grafana/provisioning:/etc/grafana/provisioning
    environment:
      GF_SECURITY_ADMIN_USER: ${ADMIN_USER:-admin}
      GF_SECURITY_ADMIN_PASSWORD: ${ADMIN_PASSWORD:-admin}
      GF_USERS_ALLOW_SIGN_UP: "false"
      VIRTUAL_HOST: ${DOMAINS}
      LETSENCRYPT_HOST: ${DOMAINS}
      LETSENCRYPT_EMAIL: ${LETSENCRYPT_EMAIL}
    restart: unless-stopped
    expose:
      - 3000
    networks:
      - monitor-net
    labels:
      org.label-schema.group: "monitoring"

    ...

Answer 1

你忘记映射端口了。

请参阅下面的 ports 示例

root@d:~# lsof -i :3000,9100,9090

COMMAND     PID USER   FD   TYPE   DEVICE SIZE/OFF NODE NAME
docker-pr 96795 root    4u  IPv6 46823321      0t0  TCP *:jetdirect (LISTEN)
docker-pr 96829 root    4u  IPv6 46831661      0t0  TCP *:websm (LISTEN)
docker-pr 96873 root    4u  IPv6 46814051      0t0  TCP *:hbci (LISTEN)

docker-compose.yml

version: '2.1'

networks:
    monitor-net:
        driver: bridge
    default:
        external:
            name: aaa

services:

  prometheus:
    image: prom/prometheus:v2.15.2
    ports:  [ "9090:9090" ]
    expose: [ "9090" ]
    networks:
        - monitor-net
    labels:
        org.label-schema.group: "monitoring"

  nodeexporter:
    image: prom/node-exporter:v0.18.1
    ports:  [ "9100:9100" ]
    expose: [ "9100" ]
    networks:
        - monitor-net
    labels:
        org.label-schema.group: "monitoring"


  grafana:
    image: grafana/grafana:6.5.3
    ports:  [ "3000:3000" ]
    expose: [ "3000" ]
    networks:
        - monitor-net
    labels:
        org.label-schema.group: "monitoring"