【问题标题】:Python XML to CSV lookup values in second dataframe and list multiple valuesPython XML to CSV 在第二个数据框中查找值并列出多个值
【发布时间】:2021-09-14 03:23:12
【问题描述】:

这与Python XML to CSV help for novice 有关。感谢@aneroid 帮助我入门。

我能够取出我的列和行,还有两个问题,我不知道要搜索什么才能找到答案。

剩余问题 1:我的 XML 中的某些属性最多有 20 个值。我想在一个逗号分隔的字符串中列出这些。请参阅下面我的示例中的“年”。

我发现了一些关于分组的东西,但我认为这不会起作用,因为我只是从我的 XML 中获取第一个值。我需要列出 XML 中的所有匹配值。另外,我的完整报告有 120 列,所以我需要列出所有要分组的列吗?

(更新开始)我研究了 lxml.etree,现在可以得到这个:

['2019', '2020']

我已经更新了下面的 Python。如果有人可以帮助将这个时间延长到 2019 年、2020 年,那就太棒了。问题 2 类似 - 只需要从字典中提取值。 (更新结束)

剩余问题 2:XML 按名称列出相关事物,然后将它们再次列为单独的“行”以及事物的附加属性。我需要在报告中包含这些附加属性之一的值。在我的示例 Python 中,我创建了第二个数据框,名为 thing_df,其中包含名称和 ID 属性。我需要将 coll_df 中的事物名称与 thing_df 中的事物名称相匹配,以获取其 thing_id 并将其添加到 coll_df。

我发现了一些关于合并数据集的内容,但这些示例似乎是用于合并我在示例中称为集合的内容,而不是用于我要查找的内容。

期望的输出:

,Collection item,ITEM-ID,ATTRIB-1,PERSON-TYPE-1-NAME,ATTRIB-2,PERSON-TYPE-2-NAME,RELATED-THING-1 id,RELATED-THING-2 IDs,Years
0,name of Item 1,item_000001,Yes,name of person 1,Yes,name of person 2,thing_000745,"thing_000783, thing_000803","2019, 2020"

Python(更新):

# -*- coding: utf-8 -*-

# Importing the required libraries
#import xml.etree.ElementTree as Xet
import lxml.etree as Xet
import pandas as pd

#Define main collection columns and rows for dataframe
coll_cols = ["Collection item", "ITEM-ID", "ATTRIB-1", "PERSON-TYPE-1-NAME" ,
        "ATTRIB-2", "PERSON-TYPE-2-NAME", "RELATED-THING-1 id",
        "RELATED-THING-2 IDs", "Years"]
coll_rows = []
#Define thing lookup dataframe columns and rows
thing_cols = ["Thing Name", "Thing ID"]
thing_rows = []

# Parsing the XML file
xmlparse = Xet.parse('sample.xml')
root = xmlparse.getroot()
for row in root:
    # Create thing lookup dataframe
    if (
            row.findtext('type') == "THING-TYPE-1" or
            row.findtext('type') == "THING-TYPE-2"
            ):
        thing_id = row.findtext("THING-ID")
        thing_name = row.findtext("name")
        thing_rows.append({"Thing Name": thing_name,
                           "Thing ID": thing_id})
        thing_df = pd.DataFrame(thing_rows, columns=thing_cols)
    # Find only collection items
    if row.findtext('type') != "COLLECTION-ITEM":
        continue
    # Define values for collection item dataframe
    name = row.findtext("name", "Missing name")
    item_id = row.findtext("ITEM-ID", "Missing item ID")
    attrib_1 = row.findtext("ATTRIB-1", "Missing attribute 1")
    p1_name = row.findtext("./PERSON-TYPE-1-NAME/result/row/name")
    attrib_2 = row.findtext("ATTRIB-2", "Missing attribute 2")
    p2_name = row.findtext("./PERSON-TYPE-2-NAME/result/row/name")
    relat_thing1 = row.xpath("./RELATED-THING-1/result/row/name/text()")
    #relat_thing1_id = look up relat_thing1 in infr_df as "Thing Name" \
    #    and return "Thing ID"
    relat_thing2 = row.xpath("./RELATED-THING-2/result/row/name/text()")
    #relat_thing2_id = look up every relat_thing2 in infr_df as "Thing Name" \
    #    and return all "Thing ID"
    years = row.xpath("./RPTD-HIST-CODE/result/row/name/text()")

    coll_rows.append({"Collection item": name,
                 "ITEM-ID": item_id,
                 "ATTRIB-1": attrib_1,
                 "PERSON-TYPE-1-NAME": p1_name,
                 "ATTRIB-2": attrib_2,
                 "PERSON-TYPE-2-NAME": p2_name,
                 #"RELATED-THING-1 id": relat_thing1_id,
                 #"RELATED-THING-2 IDs": relat_thing2_ids,
                 "Years": years
})

coll_df = pd.DataFrame(coll_rows, columns=coll_cols)

# Writing dataframe to csv
coll_df.to_csv('output.csv')

【问题讨论】:

    标签: python xml dataframe xml-parsing


    【解决方案1】:

    除了在数据框中查找“事物名称”以获取“事物 ID”之外,我已经解决了所有问题。我将发布一个新的更简单的问题。

    这是解决我的剩余问题 1 和一半剩余问题 2 的新 Python:

    # -*- coding: utf-8 -*-
    
    # Importing the required libraries
    #import xml.etree.ElementTree as Xet
    import lxml.etree as Xet
    import pandas as pd
    
    #Define main collection columns and rows for dataframe
    coll_cols = ["Collection item", "ITEM-ID", "ATTRIB-1", "PERSON-TYPE-1-NAME" ,
            "ATTRIB-2", "PERSON-TYPE-2-NAME", "RELATED-THING-1 id",
            "RELATED-THING-2 IDs", "Years"]
    coll_rows = []
    #Define thing lookup dataframe columns and rows
    thing_cols = ["Thing Name", "Thing ID"]
    thing_rows = []
    
    # Parsing the XML file
    xmlparse = Xet.parse('sample.xml')
    root = xmlparse.getroot()
    for row in root:
        # Create thing lookup dataframe
        if (
                row.findtext('type') == "THING-TYPE-1" or
                row.findtext('type') == "THING-TYPE-2"
                ):
            thing_id = row.findtext("THING-ID")
            thing_name = row.findtext("name")
            thing_rows.append({"Thing Name": thing_name,
                               "Thing ID": thing_id})
            thing_df = pd.DataFrame(thing_rows, columns=thing_cols)
        # Find only collection items
        if row.findtext('type') != "COLLECTION-ITEM":
            continue
        # Define values for collection item dataframe
        name = row.findtext("name", "Missing name")
        item_id = row.findtext("ITEM-ID", "Missing item ID")
        attrib_1 = row.findtext("ATTRIB-1", "Missing attribute 1")
        p1_name = row.findtext("./PERSON-TYPE-1-NAME/result/row/name")
        attrib_2 = row.findtext("ATTRIB-2", "Missing attribute 2")
        p2_name = row.findtext("./PERSON-TYPE-2-NAME/result/row/name")
        relat_thing1_items = row.xpath("./RELATED-THING-1/result/row/name/text()")
        if len(relat_thing1_items) > 0:
            relat_thing1 = ', '.join(relat_thing1_items)
        else:
            relat_thing1 = ""
        #relat_thing1_id = look up relat_thing1 in infr_df as "Thing Name" \
        #    and return "Thing ID"
        relat_thing2_items = row.xpath("./RELATED-THING-2/result/row/name/text()")
        if len(relat_thing2_items) > 0:
            relat_thing2 = ', '.join(relat_thing2_items)
        else:
            relat_thing2 = ""
        #relat_thing2_id = look up every relat_thing2 in infr_df as "Thing Name" \
        #    and return all "Thing ID"
        year_items = row.xpath("./RPTD-HIST-CODE/result/row/name/text()")
        if len(year_items) > 0:
            years = ', '.join(year_items)
        else:
            years = ""
    
        coll_rows.append({"Collection item": name,
                     "ITEM-ID": item_id,
                     "ATTRIB-1": attrib_1,
                     "PERSON-TYPE-1-NAME": p1_name,
                     "ATTRIB-2": attrib_2,
                     "PERSON-TYPE-2-NAME": p2_name,
                     "RELATED-THING-1 id": relat_thing1,
                     "RELATED-THING-2 IDs": relat_thing2,
                     "Years": years
    })
    
    coll_df = pd.DataFrame(coll_rows, columns=coll_cols)
    
    # Writing dataframe to csv
    coll_df.to_csv('output.csv')
    

    【讨论】:

      猜你喜欢
      • 2018-03-23
      • 1970-01-01
      • 1970-01-01
      • 1970-01-01
      • 2021-12-29
      • 2021-03-25
      • 2010-10-23
      • 1970-01-01
      • 1970-01-01
      相关资源
      最近更新 更多