Logstash 过滤器中的 Java Strace gork 模式答案

【问题标题】：Java Strace gork pattern in logstash filterLogstash 过滤器中的 Java Strace gork 模式
【发布时间】：2019-10-18 09:08:32
【问题描述】：

我有以下错误日志。我需要为它纠正 gork 模式。我的模式只能读取第一行。谁能帮我阅读另一行并将该行存储在外线字段中。这是我目前正在使用的表达式

\[%{NOTSPACE:thread}\] \[%{LOGLEVEL:loglevel}\] (?<timestamp>%{TIMESTAMP_ISO8601}) (?<logger>[A-Za-z0-9$_.]+):%{NOTSPACE:method}\(\):%{NONNEGINT:line} - %{GREEDYDATA:message}$

需要解析的日志

[pool-4-thread-1] [ERROR] 2019-06-19 12:56:14,827 com.chaipoint.boxc.dao.OrderDispenseDataDao:updateOrderStatusOnDispenseScheduler():218 - Error while updating status in scheduler
org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:627)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:906)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:930)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:313)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:318)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao.updateOrderStatusOnDispenseScheduler(OrderDispenseDataDao.java:216)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao$$FastClassBySpringCGLIB$$e49056b7.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

预期的输出是


method : updateOrderStatusOnDispenseScheduler
timestamp : 2019-06-19·12:56:14,827
logger : com.chaipoint.boxc.dao.OrderDispenseDataDao
line : 218
thread : pool-4-thread-1
message : Error·while·updating·status·in·scheduler
loglevel : ERROR
extra line : org.springframework.jdbc.CannotGetJdbcConnectionException: Could not get JDBC Connection; nested exception is com.mysql.jdbc.exceptions.jdbc4.MySQLNonTransientConnectionException: Could not create connection to database server. Attempted reconnect 3 times. Giving up.
    at org.springframework.jdbc.datasource.DataSourceUtils.getConnection(DataSourceUtils.java:80)
    at org.springframework.jdbc.core.JdbcTemplate.execute(JdbcTemplate.java:627)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:906)
    at org.springframework.jdbc.core.JdbcTemplate.update(JdbcTemplate.java:930)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:313)
    at org.springframework.jdbc.core.namedparam.NamedParameterJdbcTemplate.update(NamedParameterJdbcTemplate.java:318)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao.updateOrderStatusOnDispenseScheduler(OrderDispenseDataDao.java:216)
    at com.chaipoint.boxc.dao.OrderDispenseDataDao$$FastClassBySpringCGLIB$$e49056b7.invoke(<generated>)
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)

【问题讨论】：

标签： logstash kibana elastic-stack logstash-grok filebeat

【解决方案1】：

我的模式只能读取第一行

要读取多行，请使用：https://www.elastic.co/guide/en/logstash/current/plugins-codecs-multiline.html

例如，Java 堆栈跟踪是多行的，并且消息通常从最左侧开始，随后的每一行都缩进。这样做：

input {
  stdin {
    codec => multiline {
      pattern => "^\s"
      what => "previous"
    }
  }
}

【讨论】：