Logstash 期望 \n 处理 XML

Question

我正在尝试解析一个 XML 文件，并使用 Grok 模式从整个 XML 中只读取一个标签。

我的 grok 模式看起来像这样。它能够在正确缩进时解析 XML，因为每个结束标记后都有一个新行。但是当文件在连续标签之间没有空格时，这种模式就不起作用了。有人可以帮忙吗？

 input {
      beats {
        port => 5045
        type => 'iis'
      }

    }    
    filter {
        #ignore log comments
        if [message] =~ "^#" {
        drop {}
        }

        grok {
        patterns_dir => "./patterns"
        match => ["message", "%{DATA:extras}<LoadID%{DATA:extra}>%{DATA:ASNNumber}%{GREEDYDATA:behind}"]
        }
        date {
        match => [ "timestamp", "yyyy-MM-dd HH:mm:ss" ]
        locale => "en"
        }
        }

        Second filter
        filter {
        if "_grokparsefailure" in [tags] {
        drop { }
        } else {
        # on success remove the message field to save space
        mutate {
        remove_field => ["message", "timestamp", "extra", "extras", "behind"]
        }
        }
        }

这失败了：

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?><tns:ASNAck xmlns:tns='http://www.xyx.com/YYY/logistics/mxg/xnsds/V1_0' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><MessageHeader><MessageID>3031999515</MessageID><MessageTimeStamp>2017-09-12T06:37:36Z</MessageTimeStamp><SenderID>XBHSNS</SenderID><ReceiverID>GOLF_DAO</ReceiverID><MessageType>ACKACKACK</MessageType><CorrelationID>2d323537383935353034383933383135</CorrelationID></MessageHeader><Masterbill>G829441</Masterbill><LoadID>Jitesh555</LoadID><Accept>true</Accept><ReasonCode/><ReasonDescription/></tns:ASNAck>

这行得通：

<tns:ASNAck xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:tns="http://www.xyx.com/YYY/logistics/mxg/xnsds/V1_0">
    <MessageHeader>
        <MessageID>20170704080189</MessageID>
        <MessageTimeStamp>2017-07-04T20:17:30Z</MessageTimeStamp>
        <SenderID>KNN_DAO_MXC</SenderID>
        <ReceiverID>GOLF_DAO</ReceiverID>
        <MessageType>InboundASNAck</MessageType>
        <CorrelationID>2d383736363033383337333530313338</CorrelationID>
    </MessageHeader>
    <MasterWaybill>C211</MasterWaybill>
    <LoadID>10112275912A02</LoadID>
    <Accept>true</Accept>
    <ReasonCode>0</ReasonCode>
    <ReasonDescription/>
</tns:ASNAck>

这也有效：

<?xml version="1.0" encoding="UTF-8" standalone="yes" ?><tns:ASNAck xmlns:tns='http://www.xyx.com/YYY/logistics/mxg/xnsds/V1_0' xmlns:xsi='http://www.w3.org/2001/XMLSchema-instance'><MessageHeader><MessageID>3031999515</MessageID><MessageTimeStamp>2017-09-12T06:37:36Z</MessageTimeStamp><SenderID>XBHSNS</SenderID><ReceiverID>GOLF_DAO</ReceiverID><MessageType>ACKACKACK</MessageType><CorrelationID>2d323537383935353034383933383135</CorrelationID></MessageHeader><Masterbill>G829441</Masterbill><LoadID>Jitesh555</LoadID>
<Accept>true</Accept><ReasonCode/><ReasonDescription/></tns:ASNAck>

Answer 1

你不需要解析整个字符串，你可以抓住你感兴趣的部分。

 grok {
    patterns_dir => "./patterns"
    match => ["message", "<LoadID>%{GREEDYDATA:ASNNumber}</LoadID>"]
 }

我还删除了“extra”、“extras”、“behind”字段，因为您没有保留它们。