.net 日志从 filebeat 到 logstash 多行正确格式答案

【问题标题】：.net logs from filebeat to logstash multiline proper format.net 日志从 filebeat 到 logstash 多行正确格式
【发布时间】：2020-10-09 11:07:51
【问题描述】：

我正在尝试从 .net 应用程序中提取日志。我在将日志推送到 logstash 服务器的节点上安装了 filebeat。

日志文件：

2020-06-19 00:00:16.421 +02:00 [Error] [Band.Account.HealthCheckService] [2HB0AJ9Q9AI2O:00000001] The operation was canceled.
System.Threading.Tasks.TaskCanceledException: The operation was canceled. ---> System.IO.IOException: Unable to read data from the transport connection: Operation canceled. ---> System.Net.Sockets.SocketException: Operation canceled
   --- End of inner exception stack trace ---
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error)
   at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.GetResult(Int16 token)
   at System.Net.Security.SslStreamInternal
...

filebeat.yml 配置：

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /var/log/log*
  multiline.pattern: '^[[:space:]]'
  multiline.negate: false
  multiline.match: after
  fields_under_root: true
  fields:
    type: Band-account-log
  fields_under_root: true
  exclude_files: ['.gz$']
logging.level: info
output.logstash:
  hosts: ["elk.Band.net"]
  ssl.certificate_authorities: "/etc/pki/tls/certs/logstash-forwarder.crt"

它并没有真正从日志文件中获取所有行，并且时间戳在 kibana 中看起来是错误的。不太确定我需要修改什么。

filebeat 版本 6.2.4 (amd64)，libbeat 6.2.4*

【问题讨论】：

标签： logging logstash kibana filebeat elk

【解决方案1】：

我设法使用下面的 conf 使它工作：

filebeat.yml:

filebeat.prospectors:
- type: log
  enabled: true
  paths:
    - /var/log/log*
  multiline.pattern: '([12]\d{3}-(0[1-9]|1[0-2])-(0[1-9]|[12]\d|3[01]))'
  multiline.negate: true
  multiline.match: after
  fields_under_root: true
  fields:
    type: pirelli-account-log
  fields_under_root: true
  exclude_files: ['.gz$']
logging.level: info
output.logstash:
  hosts: ["elk.server.example"]
  ssl.certificate_authorities: "/etc/pki/tls/certs/logstash-forwarder.crt"

现在日志格式正确。

【讨论】：