出现换行时转换包含 XML 数据的 stramreader 文本时出现问题

Question

1. 我有一个日志文件，其中包含 XML 和 Jason 等混合字符串。
2. 我必须提取它们中的每一个以获得所需的输出。
3. xml_string 包含可以换行的元素。
4. 我首先读取所有行，然后使用 Environment.NewLine 将它们断开以获取 log_file_lines，然后在 IEnumerable 中为 xml_lines 和 jason_lines 获取行。
5. 但是，当 xml_lines 中有新行时，就会出现问题。这意味着我在字段中得到了一半或未格式化的 xml_ 行。
6. 在正则表达式中有什么方法或从 log_file_string 文本本身中删除这些行，然后将 t 传递给 log_file_lines ？
7. 另一个选项可以是使用正则表达式从 log_file_string 中的节点和 xml_lines 之间获取数据，而无需循环，因为数据是 3 MB 文件:(

log_file_string 的显示方式如下：

2020-06-10T10:58:07.0792762Z [data_type_jason] {"person_id":"101", "order_id":"123"}
2020-06-12T10:58:07.0792762Z [data_type_xml] <?xml version="1.0"?><persons><person id = "101"><name>"Thomas Edison"</name><age>"35"</age><phone>"7777777777"</phone><address>"62  Ross Road, 
    MARSHAM, NR10 6EA"</address><country>"England"</country></person></persons>
2020-06-13T10:58:07.0792762Z [data_type_jason]  {"person_id":"102", "order_id":"140"}
2020-06-14T10:58:07.0792762Z [data_type_xml]<?xml version="1.0"?><persons><person id = "102"><name>"Louis Pasture"</name><age>"40"</age><phone>"99999999"</phone><address>"145  Thames Street, BOOSBECK, TS12 1AN"</address><country>"England"</country></person></persons>

这是完整的原型宝贝：

using System;
using System.Collections.Generic;
using System.Data;
using System.IO;
using System.Linq;
using System.Text;
using System.Windows.Forms;
using System.Xml.Linq;
namespace Test
{
    public partial class Form1 : Form
    {
        public Form1()
        {
            InitializeComponent();
        }//Form1

        private void Form1_Load(object sender, EventArgs e)
        {
            String file_folder = @"X:\VS 2019\C Sharp\Test";
            String file_path = Path.Combine(file_folder, "log_file.txt");
            process_log_file_data(file_folder, file_path);
        }//Form1_Load

        private void process_log_file_data(String file_folder, String file_path)
        {
            String log_file_string = read_all_lines_from_file(file_folder, file_path);
            String[] log_file_lines = log_file_string.Split(new String[] { Environment.NewLine }, StringSplitOptions.None);

            //The other option can be getting data between just to nodes <persons> and </persons> in xml_lines from the log_file_string using regex but am not regex savvy :(
            IEnumerable<String> xml_lines = from line in log_file_lines
                                            where line.Contains("data_type_xml")
                                            select line;

            IEnumerable<String> jason_lines = from line in log_file_lines
                                              where line.Contains("data_type_jason")
                                              select line;

            XDocument xml_document = new XDocument(new XDeclaration("1.0", "utf-8", "yes"), new XElement("xml_data"));

            foreach (var xml_line in xml_lines)
            {
                String line = xml_line.Split(new String[] { "[data_type_xml]" }, StringSplitOptions.None)[xml_line.Split(new String[] { "[data_type_xml]" }, StringSplitOptions.None).GetUpperBound(0)].Trim();

            //here is the issue < persond id = "101" >< address > as the  there is a line break in log_file_lines  the xml_line = 2020-06-12T10:58:07.0792762Z [data_type_xml] <?xml version="1.0"?><person><person id = "101"><name>"Thomas Edison"<name><age>"35"</age><phone>"7777777777"</phone><address>"62  Ross Road
            XDocument temp_xml_document = XDocument.Parse(line); //Unexpected end of file has occurred. The following elements are not closed: address, person, persons. Line 1, position 144.'
            }

            foreach (var jason_line in jason_lines)
            {
                //do something
            }
        }//process_log_file_data(String file_folder, String file_path)

        private String read_all_lines_from_file(String file_folder, String file_path)
        {
            FileInfo file_info = new FileInfo(file_path);
            if ((!file_info.Exists) || (file_info.Length == 0))
            {
                return String.Empty;
            }

            FileStream file_stream; StreamReader stream_reader; UTF8Encoding utf8_encoding; String file_text;
            file_stream = new FileStream(file_path, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
            utf8_encoding = new UTF8Encoding(false);
            stream_reader = new StreamReader(file_stream, utf8_encoding);
            file_text = stream_reader.ReadToEnd();
            stream_reader.Close();
            file_stream.Close();
            return file_text;
        }//read_all_lines_from_file

    }//Form1 : Form

}//Test

Answer 1

使用递归和流式阅读器可以实现如下。 想法是同时读取两条 readlines，条件很少...

class Program
{
    static void Main(string[] args)
    {
        using (StreamReader r = new StreamReader("filename"))
        {
            while (!r.EndOfStream)
            {
                var fline = r.ReadLine();
                string sline = "";
                if (!r.EndOfStream)
                    sline = r.ReadLine();

                RecursivelyParse(r,ref fline, ref sline);
            }
        }
    }

    private static void RecursivelyParse(StreamReader r, ref string fline, ref string sline)
    {
        if (fline.Contains("data_type_xml"))
        {
            if (!(sline.Contains("data_type_xml") || sline.Contains("data_type_jason")) && sline != "")
            {
                fline += sline;
                sline = "";
                //Next line also a part of xml
                //parsing loggic of line containing  xml 
            }
            else
            {
                //parsing loggic of line containing  xml 
            }
        }
        else if (fline.Contains("data_type_jason"))
        {
            //parsing loggic of line containing  jason 
        }

        if (sline.Contains("data_type_xml"))
        {
            if (!r.EndOfStream)
                fline = r.ReadLine();
            else
                fline = "";

            if (!(fline.Contains("data_type_xml") || fline.Contains("data_type_jason")) && fline != "")
            {
                sline += fline;
                fline = "";
                //Next line also a part of xml
                //parsing loggic of line containing  xml 
            }
            else
            {
                //parsing loggic of line containing  xml 
            }
        }
        else if (sline.Contains("data_type_jason"))
        {
            //parsing loggic of line containing  jason 
        }

        while (!r.EndOfStream)
        {
            fline = r.ReadLine();

            if (!r.EndOfStream)
                sline = r.ReadLine();
            else 
                sline = "";

            RecursivelyParse(r, ref fline, ref sline);
        }
    }

}

Answer 2

解析日志条目：

试试这样的：

public static IList<string> ParseLogs(string log_file_string)
{
    // The pattern of the beginning of each log entry
    var logprefixPattern = @"\d{4}-\d{2}-\d{2}T\d{2}:\d{2}:\d{2}\.\d+Z \[data_type_(jason|xml)\] ";

    // Replace it with a symbol
    var splitPattern = "[log_entry]";
    var replaced = Regex.Replace(log_file_string, logprefixPattern, splitPattern);

    // Split the string with that symbol
    var splited = replaced.Split(splitPattern);

    // Now you get the list of logs.
    return splited;
}

我的想法是你读取所有内容并存储到变量log_file_string中，你可以查找每个日志的模式而不是逐行读取。

看起来每个日志条目都以相同的模式开头，因此您可以按该模式拆分日志条目。

如果要将项目分为 2 组：xml 日志和 json 日志，您可以将拆分的结果一一循环，并通过 e.g. 确定。开头字符（{ 是 json，< 是 xml）。

解析日志条目中的 XML

假设你想把XML解析成一个json字符串列表，每个字符串都是一个人，你可以使用Newtonsoft.Json：

public static IList<string> ParseXml(string xmlStr)
{
    XmlDocument doc = new XmlDocument();
    doc.LoadXml(xmlStr);

    // assuming that each entry contains at least 1 person, and you want to extract those persons
    var ppl = doc.GetElementsByTagName("person");
    var results = new List<string>{};
    foreach(XmlNode p in ppl)
    {
        string jsonStr = Newtonsoft.Json.JsonConvert.SerializeXmlNode(p);
        results.Add(jsonStr);       
    }
    return results;
}

为简单起见，我只在此处返回 json 字符串。您可以将jsonStr 反序列化为您定义的某些 C# 模型。