带有json数据的流利过滤器正则表达式答案

【问题标题】：fluentd filter regexp with json data带有json数据的流利过滤器正则表达式
【发布时间】：2020-07-20 12:31:58
【问题描述】：

我正在尝试仅过滤（保留）经过 oauth 身份验证的审核日志，但它不起作用。当我将模式设置为 /.*/ 时，所有内容都会出现在 Kibana 中。如果我设置 /oauth/ 或 /.*oauth.*/ 什么都不会出现。我做错了什么？

日志文件中我要过滤的行：

{"kind":"Event","apiVersion":"audit.k8s.io/v1beta1","metadata":{"creationTimestamp":"2020-07-17T20:06:49Z"},"level":"Metadata","timestamp":"2020-07-17T20:06:49Z","auditID":"cf56d61e-30b3-486c-a513-6bd9e96fb592","stage":"ResponseComplete","requestURI":"/api/v1/namespaces/openshift-logging/pods?limit=500","verb":"list","user":{"username":"user","uid":"388e0232-c5bb-11ea-904d-7a59592b634f","groups":["system:authenticated:oauth","system:authenticated"],"extra":{"scopes.authorization.openshift.io":["user:full"]}},"sourceIPs":["10.0.72.20"],"objectRef":{"resource":"pods","namespace":"openshift-logging","apiVersion":"v1"},"responseStatus":{"metadata":{},"code":200},"requestReceivedTimestamp":"2020-07-17T20:06:49.918391Z","stageTimestamp":"2020-07-17T20:06:49.921475Z","annotations":{"authorization.k8s.io/decision":"allow","authorization.k8s.io/reason":"RBAC: allowed by ClusterRoleBinding \"registry-controller\" of ClusterRole \"cluster-admin\" to User \"user\""}}

流利的配置：

<source>
  @type tail
  @id in_tail_audit_logs
  multiline_flush_interval 5s
  path "/var/lib/origin/audit-ocp.log"
  tag "ocp-audit"
  <parse>
    @type "json"
    time_format "%Y-%m-%dT%T.%L%Z"
    time_type string
  </parse>
</source>
<filter ocp-audit>
  @type grep
  <regexp>
    key user.groups
    pattern /oauth/
  </regexp>
</filter>

【问题讨论】：

试试：key user 和 pattern /.*groups.*oauth/
@Azeem 非常感谢。成功了。
不客气！

标签： json filter fluentd

【解决方案1】：

试试这个配置grep:

<regexp>
  key user
  pattern /.*groups.*oauth/
</regexp>

或者，您可以像这样安装和配置fluent-plugin-json：

<filter ocp-audit>
  @type json
  @id json_filter

  <check>
    pointer /user/groups/0    # point to 0th index of groups array
    pattern /.*:oauth/
  </check>
</filter>

【讨论】：