filebeat + kubernetes + elasticsearch 不保存特定字段答案

【问题标题】：filebeat + kubernetes + elasticsearch not save specific fieldsfilebeat + kubernetes + elasticsearch 不保存特定字段
【发布时间】：2020-01-23 14:07:42
【问题描述】：

我创建了一个命名空间来获取带有 filebeats 的日志并保存到 elasticsearch。为什么不将关于 Kubernetes 的字段保存在 elasticsearch 上如何示例？

Kubernetes fields

        "kubernetes" : {
            "labels" : {
              "app" : "MY-APP",
              "pod-template-hash" : "959f54cd",
              "serving" : "true",
              "version" : "1.0",
              "visualize" : "true"
            },
            "pod" : {
              "uid" : "e20173cb-3c5f-11ea-836e-02c1ee65b375",
              "name" : "MY-APP-959f54cd-lhd5p"
            },
            "node" : {
              "name" : "ip-xxx-xx-xx-xxx.ec2.internal"
            },
            "container" : {
              "name" : "istio"
            },
            "namespace" : "production",
            "replicaset" : {
              "name" : "MY-APP-959f54cd"
            }
          }

目前是这样保存的：

      "_source" : {
          "@timestamp" : "2020-01-23T12:33:14.235Z",
          "ecs" : {
            "version" : "1.0.0"
          },
          "host" : {
            "name" : "worker-node1"
          },
          "agent" : {
            "hostname" : "worker-node1",
            "id" : "xxxxx-xxxx-xxx-xxxx-xxxxxxxxxxxxxx",
            "version" : "7.1.1",
            "type" : "filebeat",
            "ephemeral_id" : "xxxx-xxxx-xxxx-xxxxxxxxxxxxx"
          },
          "log" : {
            "offset" : xxxxxxxx,
            "file" : {
              "path" : "/var/lib/docker/containers/xxxx96ec2bfd9a3e4f4ac83581ad90/7fd55e1249aa009df3f8e3250c967bbe541c9596xxxxxac83581ad90-json.log"
            }
          },
          "stream" : "stdout",
          "message" : "xxxxxxxx",
          "input" : {
            "type" : "docker"
          }
        }

关注我的 filebeat.config：

apiVersion: v1
kind: ConfigMap
metadata:
  name: filebeat-config
  namespace: kube-system
  labels:
    k8s-app: filebeat
data:
  filebeat.yml: |-
    filebeat.config:
      inputs:
        # Mounted `filebeat-inputs` configmap:
        path: ${path.config}/inputs.d/*.yml
        # Reload inputs configs as they change:
        reload.enabled: false
        multiline.pattern: '^[[:space:]]'
        multiline.negate: false
        multiline.match: after
      modules:
        path: ${path.config}/modules.d/*.yml
        # Reload module configs as they change:
        reload.enabled: false

    # To enable hints based autodiscover, remove `filebeat.config.inputs` configuration and uncomment this:
    #filebeat.autodiscover:
    #  providers:
    #    - type: kubernetes
    #      hints.enabled: true

    processors:
      - add_cloud_metadata:
      - add_kubernetes_metadata:

    cloud.id: ${ELASTIC_CLOUD_ID}
    cloud.auth: ${ELASTIC_CLOUD_AUTH}

    output.elasticsearch:
      hosts: ['${ELASTICSEARCH_HOST:elasticsearch}:${ELASTICSEARCH_PORT:9200}']
      protocol: "http"
    setup.ilm.enabled: false
    ilm.enabled: false
    xpack.monitoring:
      enabled: true

DamemonSet 如下图所示：

apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: filebeat
  namespace: kube-system
  labels:
    k8s-app: filebeat
spec:
  template:
    metadata:
      labels:
        k8s-app: filebeat
    spec:
      serviceAccountName: filebeat
      hostNetwork: true
      terminationGracePeriodSeconds: 30
      containers:
        - name: filebeat
          image: docker.elastic.co/beats/filebeat-oss:7.1.1
          args: [
            "-c", "/etc/filebeat.yml",
            "-e",
          ]
          env:
            - name: ELASTICSEARCH_HOST
              value: xxxxxxxxxxxxx
            - name: ELASTICSEARCH_PORT
              value: "9200"
          securityContext:
            runAsUser: 0
            # If using Red Hat OpenShift uncomment this:
            #privileged: true
          resources:
            limits:
              memory: 200Mi
            requests:
              cpu: 100m
              memory: 100Mi
          volumeMounts:
            - name: config
              mountPath: /etc/filebeat.yml
              readOnly: true
              subPath: filebeat.yml
            - name: inputs
              mountPath: /usr/share/filebeat/inputs.d
              readOnly: true
            - name: data
              mountPath: /usr/share/filebeat/data
            - name: varlibdockercontainers
              mountPath: /var/lib/docker/containers
              readOnly: true
      volumes:
        - name: config
          configMap:
            defaultMode: 0600
            name: filebeat-config
        - name: varlibdockercontainers
          hostPath:
            path: /var/lib/docker/containers
        - name: inputs
          configMap:
            defaultMode: 0600
            name: filebeat-inputs
        # data folder stores a registry of read status for all files, so we don't send everything again on a Filebeat pod restart
        - name: data
          hostPath:
            path: /var/lib/filebeat-data
            type: DirectoryOrCreate

在将配置应用到 kubernetes 之前，我确实删除了 elasticsearch 的注册表文件节拍。

【问题讨论】：

您能否编辑您的问题，更好地解释您做了什么以及您想要实现什么？
你的问题到底是什么？
我正在使用 filebeat 在 elasticsearch 上保存字段，我需要保存有关我的 kubernetes 的这些字段。 elastic.co/guide/en/beats/filebeat/master/…
您的 ConfigMap 似乎在容器的日志中缺少 paths:。类似：/var/log/containers/*${data.kubernetes.container.id}.log。请将您的配置与this one 进行比较。
你成功了吗？

标签： docker elasticsearch kubernetes kibana filebeat

【解决方案1】：

正如我在评论中所述。看起来您的 ConfigMap 缺少容器日志中的 paths:。应该是这样的：

       type: container
       paths:
         - /var/log/containers/*${data.kubernetes.container.id}.log

将您的配置文件与this one 进行比较。

希望对你有帮助。

【讨论】：

【解决方案2】：

我遇到了同样的问题，我通过从 DaemonSet 中删除 hostNetwork: true 配置来解决。这意味着 pod 名称与节点名称相同。查看filebeat启动日志，可以看到这个。

【讨论】：