如何仅打印与python中的模式匹配的数据文件的特定部分

Question

我想使用 python 从文件中打印某些数据块。基本上它应该作为解析器工作，并且只输出符合我的标准的块。 我的文件包含呼叫中心的日志。我想要以“####”开头并以"</soap:Body>>" 结尾的部分，但它还应该在我的文件中包含一个称为msisdn 的特定数字："<msisdn>any number</msisdn>"

而且文件有点大。因此，当我执行 readlines() 时，我无法使用 for i,data in enumerate(line) 的正则表达式 这里数据被拆分，我无法搜索我需要的整个块。

文件的部分在这里：

####<Jun 4, 2016 12:05:50 PM IST> <Debug> <MessagingBridgeRuntimeVerbose> <ggneai29> <AircelESB_MS1> <[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1465022150722> <BEA-000000> <Bridge NPGBridge doTrigger(): state = 4 stopped = false> 
####<Jun 4, 2016 12:05:50 PM IST> <Error> <ALSB Logging> <ggneai29> <AircelESB_MS1> <[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1465022150886> <BEA-000000> < [PipelinePairNode1, PipelinePairNode1_request, CreateVASReportingStage, REQUEST] *** CreateVASWrapper Reprting Stage VAS V-3.0 ***: <soap:Body xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <VASProxyType xmlns="http://xmlns.aircel.com/AircelTransformation/ProxyService/OrderProxy/1.0/CreateVASSubscriptionConsumerSchema">
    <TransactionId>DATA030620160431128801011429ADD</TransactionId>
    <msisdn>8801011429</msisdn>
    <productCode>DATA</productCode>
    <action>ADD</action>
    <IMSI>405801124044563</IMSI>
    <SubsType>PrePaid</SubsType>
  </VASProxyType>
</soap:Body>> 
####<Jun 4, 2016 12:05:50 PM IST> <Error> <ALSB Logging> <ggneai29> <AircelESB_MS1> <[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'> <<anonymous>> <> <> <1465022150889> <BEA-000000> < [PipelinePairNode1, PipelinePairNode1_request, Authentication, REQUEST] ***REQUEST FOR VAS V-3.0 ****: <soap:Body xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  <VASProxyType xmlns="http://xmlns.aircel.com/AircelTransformation/ProxyService/OrderProxy/1.0/CreateVASSubscriptionConsumerSchema">
    <TransactionId>DATA030620160431128801011429ADD</TransactionId>

输出应该是：

* CreateVASWrapper Reprting阶段 VAS V-3.0 *：

DATA030620160431128801011429添加 8801011429 数据 添加 405801124044563 预付 >

请帮忙！

Answer 1

正如 cmets 所建议的：您的 XML 无效。最好确保 XML 有效，然后使用 [etree][1] 或 [Beautiful Soup][2] 之类的解析器。

但如果你还是想使用正则表达式，你可以试试：

import re

mytext = [
    '####<Jun 4, 2016 12:05:50 PM IST> <Debug> <MessagingBridgeRuntimeVerbose> <ggneai29> <AircelESB_MS1> <[ACTIVE] ExecuteThread: \'13\' for queue: \'weblogic.kernel.Default (self-tuning)\'> <<WLS Kernel>> <> <> <1465022150722> <BEA-000000> <Bridge NPGBridge doTrigger(): state = 4 stopped = false>',
    '####<Jun 4, 2016 12:05:50 PM IST> <Error> <ALSB Logging> <ggneai29> <AircelESB_MS1> <[ACTIVE] ExecuteThread: \'13\' for queue: \'weblogic.kernel.Default (self-tuning)\'> <<anonymous>> <> <> <1465022150886> <BEA-000000> < [PipelinePairNode1, PipelinePairNode1_request, CreateVASReportingStage, REQUEST] *** CreateVASWrapper Reprting Stage VAS V-3.0 ***: <soap:Body xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">',
    '<VASProxyType xmlns="http://xmlns.aircel.com/AircelTransformation/ProxyService/OrderProxy/1.0/CreateVASSubscriptionConsumerSchema">',
    '    <TransactionId>DATA030620160431128801011429ADD</TransactionId>',
    '    <msisdn>8801011429</msisdn>',
    '    <productCode>DATA</productCode>',
    '    <action>ADD</action>',
    '    <IMSI>405801124044563</IMSI>',
    '    <SubsType>PrePaid</SubsType>',
    '</VASProxyType>',
    '</soap:Body>',
    '<Jun 4, 2016 12:05:50 PM IST> <Error> <ALSB Logging> <ggneai29> <AircelESB_MS1> <[ACTIVE] ExecuteThread: \'13\' for queue: \'weblogic.kernel.Default (self-tuning)\'> <<anonymous>> <> <> <1465022150889> <BEA-000000> < [PipelinePairNode1, PipelinePairNode1_request, Authentication, REQUEST] ***REQUEST FOR VAS V-3.0 ****: <soap:Body xmlns:soap="http://schemas.xmlsoap.org/soap/envelope/" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">',
    '    <VASProxyType xmlns="http://xmlns.aircel.com/AircelTransformation/ProxyService/OrderProxy/1.0/CreateVASSubscriptionConsumerSchema">',
    '        <TransactionId>DATA030620160431128801011429ADD</TransactionId>',
]

searches = [
    {
       "if_in": "<[ACTIVE] ExecuteThread:",
       "search": "<\[ACTIVE[^<>]+> <<WLS Kernel>> <> <> <\d+>",
    },
    {
        "if_in": "PipelinePairNode1, PipelinePairNode1_request, Create",
        "search": "< \[PipelinePairNode1, PipelinePairNode1_request, Create[^\[\]]+\]",
    },
    {
        "if_in": "CreateVASWrapper Reprting Stage VAS",
        "search": "CreateVASWrapper Reprting Stage VAS[^*]+",
    },
    {
        "if_in": "<TransactionId>",
        "search": "(?<=<TransactionId>)[^<>]+",
    },
    {
        "if_in": "<msisdn>",
        "search": "(?<=<msisdn>)[^<>]+",
    },
    {
        "if_in": "<action>",
        "search": "(?<=<action>)[^<>]+",
    },
    {
        "if_in": "<IMSI>",
        "search": "(?<=<IMSI>)[^<>]+",
    },
    {
        "if_in": "<SubsType>",
        "search": "(?<=<SubsType>)[^<>]+",
    },
]

result = ""
found_once = []

for item in mytext:
    for search in searches:
        if search['if_in'] in item and search['if_in'] not in found_once:
            f = re.findall(search['search'], item)
            if f:
                result += f[0] + " "
                found_once.append(search['if_in'])

print result

如果您想查找其他内容，请将其添加到searches。

结果是：

<[ACTIVE] ExecuteThread: '13' for queue: 'weblogic.kernel.Default (self-tuning)'> <<WLS Kernel>> <> <> <1465022150722> < [PipelinePairNode1, PipelinePairNode1_request, CreateVASReportingStage, REQUEST] CreateVASWrapper Reprting Stage VAS V-3.0  DATA030620160431128801011429ADD 8801011429 ADD 405801124044563 PrePaid

Answer 2

处理此类问题的规范方法是编写某种“基于事件”的解析器（如 SAX xml 解析器...）：您的解析器逐行读取文件（您不需要阅读整个内存中的内容），根据您自己的规则扫描行（这就是您可能想要使用正则表达式的地方，但有时纯字符串方法同样有效）并根据行内容发出给定的“事件”（将被处理通过回调方法）与相关数据。

在您的情况下，您将有一个事件用于开始一个有趣的数据块（以“####”开头的行），另一个用于包含 xml 数据的行，以及一个用于块的最后一行的事件（包含 "") 的行 - 像这样：

class Parser(object):

    def parse(self, logfile):
        self.in_block = False
        for line in logfile:
            if self.is_block_start(line):
                self.in_block = True
                self.handle_block_start(line)
            elif self.in_block:
                if self.is_data(line):
                    self.handle_data(line)
                elif self.is_block_end(line):
                    self.in_block = False
                    self.handle_block_end(line)
            else:
                continue

    def is_block_start(self):
        # your code here

    def is_data(self):
        # your code here

    def is_block_end(self):
        # your code here

    def handle_block_start(self, line):
        # your code here

    def handle_data(self, line):
        # your code here

    def handle_block_end(self, line):
        # your code here