如果数据框中包含来自另一个数据框中的任何元素，如何在数据框中分配类别？

Question

我有两张 Excel 表格。一个包含摘要，另一个包含具有潜在过滤词的类别。如果第二个数据框中的任何元素匹配，我需要为第一个数据框分配类别。

我试图扩展第二个数据框中的列表，并通过将术语与第一个数据框中的任何单词进行匹配来进行映射。

测试数据。

import pandas as pd

data1 = {'Bucket':['basket', 'bushel', 'peck', 'box'], 'Summary':['This is a basket of red apples. They are sour.', 'We found a bushel of fruit. They are red and sweet.', 'There is a peck of pears that taste sweet. They are very green.', 'We have a box of plums. They are sour and have a great color.']}

data2 = {'Category':['Fruit', 'Color'], 'Filters':['apple, pear, plum, grape', 'red, purple, green']}

df1 = pd.DataFrame(data1)

df2 = pd.DataFrame(data2)

---

print(df1)

   Bucket                                            Summary
0  basket     This is a basket of red apples. They are sour.
1  bushel  We found a bushel of fruit. They are red and s...
2    peck  There is a peck of pears that taste sweet. The...
3     box  We have a box of plums. They are sour and have...

---

print(df2)

  Category                   Filters
0    Fruit  apple, pear, plum, grape
1    Color        red, purple, green

---

这行脚本将表中的 Category 列转换为列表以供以后使用。

category_list =  df2['Category'].values

category_list = list(set(category_list))

尝试匹配文本。

for item in category_list:

    item = df2.loc[df2['Category'] == item]

    filter_list =  item['Filters'].values

    filter_list = list(set(filter_list))

    df1 = df1 [df1 ['Summary'].isin(filter_list)] 

我希望第一个数据框具有分配给它的类别，用逗号分隔。

结果：

Bucket      Category                                            Summary
0  basket  Fruit, Color     This is a basket of red apples. They are sour.
1  bushel         Color  We found a bushel of fruit. They are red and s...
2    peck  Fruit, Color  There is a peck of pears that taste sweet. The...
3     box         Fruit  We have a box of plums. They are sour and have...

我希望这很清楚。我已经用头撞了一个星期了。

提前谢谢你

Answer 1

使用pandas.Series.str.contains 循环检查过滤器：

df2['Filters']=[key.replace(' ','') for key in df2['Filters']]
df2['Filters']=df2['Filters'].apply(lambda x : x.split(','))
Fruit=pd.DataFrame([df1['Summary'].str.contains(key) for key in df2.set_index('Category')['Filters']['Fruit']]).any()
Color=pd.DataFrame([df1['Summary'].str.contains(key) for key in df2.set_index('Category')['Filters']['Color']]).any()
print(Fruit)
print(Color)

0     True
1    False
2     True
3     True
dtype: bool 

0     True
1     True
2     True
3    False
dtype: bool

---

然后使用np.where 和Series.str.cat 来获取您的数据帧输出：

df1['Fruit']=np.where(Fruit,'Fruit','')
df1['Color']=np.where(Color,'Color','')
df1['Category']=df1['Fruit'].str.cat(df1['Color'],sep=', ')
df1=df1[['Bucket','Category','Summary']]
print(df1)

---

   Bucket      Category                                            Summary
0  basket  Fruit, Color     This is a basket of red apples. They are sour.
1  bushel       , Color  We found a bushel of fruit. They are red and s...
2    peck  Fruit, Color  There is a peck of pears that taste sweet. The...
3     box       Fruit,   We have a box of plums. They are sour and have...

---

到 n 个类别过滤器：

df2['Filters']=[key.replace(' ','') for key in df2['Filters']]
df2['Filters']=df2['Filters'].apply(lambda x : x.split(','))
Categories=[pd.Series(np.where(( pd.DataFrame([df1['Summary'].str.contains(key) for key in df2.set_index('Category')['Filters'][category_filter]]).any() ),category_filter,'')) for category_filter in df2['Category']]
df1['Category']=Categories[0].str.cat(Categories[1:],sep=', ')
df1=df1.reindex(columns=['Bucket','Category','Summary'])
print(df1)

   Bucket      Category                                            Summary
0  basket  Fruit, Color     This is a basket of red apples. They are sour.
1  bushel       , Color  We found a bushel of fruit. They are red and s...
2    peck  Fruit, Color  There is a peck of pears that taste sweet. The...
3     box       Fruit,   We have a box of plums. They are sour and have...

Answer 2

这是我尝试使用正则表达式模式和 pandas 字符串替换功能。第一个过滤器用“|”连接获取使用 findall 匹配的正则表达式模式，该模式将匹配放在相应组的元组中，然后用于查找匹配的类别

import pandas as pd

data1 = {'Bucket':['basket', 'bushel', 'peck', 'box'], 'Summary':['This is a basket of red apples. They are sour.', 'We found a bushel of fruit. They are red and sweet.', 'There is a peck of pears that taste sweet. They are very green.', 'We have a box of plums. They are sour and have a great color.']}

data2 = {'Category':['Fruit', 'Color'], 'Filters':['apple, pear, plum, grape', 'red, purple, green']}

df1 = pd.DataFrame(data1)

df2 = pd.DataFrame(data2)

pat = df2.Filters.str.replace(", ", "|").str.replace("(.*)", "(\\1)").str.cat(sep="|")

found = df1.Summary.str.findall(pat) \
 .apply(lambda x: [i for m in x for i, k in enumerate(m) if k!=""])

## for pandas 0.25 and above
# found= found.explode()

# for pandas below 0.25
found = found.apply(lambda x: pd.Series(x)).unstack().reset_index(level=0, drop=True).dropna()

found.name = "Cat_ID"

result = df1.merge(found, left_index=True, right_index=True) \
    .merge(df2["Category"], left_on="Cat_ID", right_index=True).drop("Cat_ID", axis=1)


result = result.groupby(result.index).agg({"Bucket":"min", "Summary": "min", "Category": lambda x: ", ".join(x)})

result