使用自定义标签解析文件答案

【问题标题】：Parsing file with custom tags使用自定义标签解析文件
【发布时间】：2019-12-16 09:54:49
【问题描述】：

我有类似这样结构的文件

<review>
<unique_id>
6301968972:just_as_true_today!:gary_d._thomas_"brass_crass"
</unique_id>
<unique_id>
5139
</unique_id>
<asin>
6301968972
</asin>
<product_name>
Hucksters: Video: Clark Gable,Deborah Kerr,Sydney Greenstreet,Adolphe Menjou,Ava Gardner,Keenan Wynn,Edward Arnold,Aubrey Mather,Richard Gaines,Frank Albertson,Douglas Fowley,Clinton Sundberg,Gloria Holden,Connie Gilchrist,Kathryn Card,Lillian Bronson,Vera Marshe,Ralph Bunker,Virginia Dale,Jimmy Conlin,Jack Conway
</product_name>
<product_type>
video
</product_type>
<product_type>
video
</product_type>
<helpful>
1 of 1
</helpful>
<rating>
5.0
</rating>
<title>
Just as true today!
</title>
<date>
August 4, 2000
</date>
<reviewer>
Gary D. Thomas "Brass Crass"
</reviewer>
<reviewer_location>
Studio City, CaliforniA
</reviewer_location>
<review_text>
Picture this: Clark Gable and the whole passle of MGM stars and starlets!  Spins a great story, and all this in 1947, could be today, except they  dressed better. You will not forget the characterizations, especially  Sydney Greenstreet, eat your heart out Eddie Murphy, just a marvy little  film for all ages (God I love cliches!
</review_text>
</review>

我希望它解析并将其转换为 datadframe。有哪些简单的方法可以做到这一点？

我尝试过 XML 和 HTML 解析器，但没有成功。该文件包含上述 sn-p 的重复，如果那里有任何图书馆，请提及。

谢谢

【问题讨论】：

Python xml parser 对我来说很好用。但是，有重复的标签，例如 unique_id 或 product_type - 您打算如何处理这些标签？
解析器给了我错误：junk after document element: line x, column y。我只需要同时使用"unique_id"s 和product type，因为我不能跳过它们
从您发布的示例中，我无法重现此内容。下一个问题是：如果您有重复的条目，那么从中创建 pandas DataFrame 将很困难。
也许您的问题是您的根目录中有多个子句 - 这在 xml 中不起作用；您将需要将它们“封装”在一个通用标签下，例如以开始您的 xml 文件并以结束（“内容”在这里是任意的）。
很高兴为您提供帮助。如果您在使用中有任何问题，请告诉我

标签： python dataframe parsing tags

【解决方案1】：

如果使用以下代码，需要安装simple_scrapy

pip install simple_scrapy

from simplified_scrapy.simplified_doc import SimplifiedDoc 
html = '''<review><unique_id>6301968972:just_as_true_today!:gary_d._thomas_"brass_crass"</unique_id><unique_id>5139
</unique_id><asin>6301968972</asin><product_name>Hucksters: Video: Clark Gable,Deborah Kerr,Sydney Greenstreet,Adolphe Menjou,Ava Gardner,Keenan Wynn,Edward Arnold,Aubrey Mather,Richard Gaines,Frank Albertson,Douglas Fowley,Clinton Sundberg,Gloria Holden,Connie Gilchrist,Kathryn Card,Lillian Bronson,Vera Marshe,Ralph Bunker,Virginia Dale,Jimmy Conlin,Jack Conway
</product_name><product_type>video</product_type><product_type>video</product_type><helpful>1 of 1</helpful><rating>5.0</rating><title>Just as true today!</title><date>August 4, 2000</date><reviewer>Gary D. Thomas "Brass Crass"</reviewer><reviewer_location>Studio City, CaliforniA
</reviewer_location><review_text>Picture this: Clark Gable and the whole passle of MGM stars and starlets!  Spins a great story, and all this in 1947, could be today, except they  dressed better. You will not forget the characterizations, especially  Sydney Greenstreet, eat your heart out Eddie Murphy, just a marvy little  film for all ages (God I love cliches!
</review_text></review>
'''
doc = SimplifiedDoc(html)
doc = doc.getElementByTag('review')
items = doc.getChildren() # get all
for item in items:
  print (item.tag,item.text)

print (doc.unique_id.html) # get one
items = doc.unique_ids # get all nodes of a label
for item in items:
  print (item.tag,item.text)

【讨论】：

【解决方案2】：

你可以使用BeautifulSoup来解决。

from bs4 import BeautifulSoup as BS4

my_text = "<review><unique_id>6301968972:just_as_true_today!:gary_d._thomas_"brass_crass"</unique_id><unique_id>5139</unique_id><asin>6301968972</asin><product_name>Hucksters: Video: Clark Gable,Deborah Kerr,Sydney Greenstreet,Adolphe Menjou,Ava Gardner,Keenan Wynn,Edward Arnold,Aubrey Mather,Richard Gaines,Frank Albertson,Douglas Fowley,Clinton Sundberg,Gloria Holden,Connie Gilchrist,Kathryn Card,Lillian Bronson,Vera Marshe,Ralph Bunker,Virginia Dale,Jimmy Conlin,Jack Conway</product_name><product_type>video</product_type><product_type>video</product_type><helpful>1 of 1</helpful><rating>5.0</rating><title>Just as true today!</title><date>August 4, 2000</date><reviewer>Gary D. Thomas "Brass Crass"</reviewer><reviewer_location>Studio City, CaliforniA</reviewer_location><review_text>Picture this: Clark Gable and the whole passle of MGM stars and starlets!  Spins a great story, and all this in 1947, could be today, except they  dressed better. You will not forget the characterizations, especially  Sydney Greenstreet, eat your heart out Eddie Murphy, just a marvy little  film for all ages (God I love cliches!</review_text></review>"


soup = BS4(my_text)

print(soup.prettify())

l_unique = soup.findAll("unique_id")

for l in l_unique:
    print(l.text)

它会打印出来

6301968972:just_as_true_today!:gary_d._thomas_"brass_crass"
5139

您可以找到 BeautifulSoup 文档here

【讨论】：