将 pandas 列中的关键字与另一个元素列表匹配

Question

我有一个熊猫数据框：

word_list
['nuclear','election','usa','baseball']
['football','united','thriller']
['marvels','hollywood','spiderman']
....................
....................
....................

我还有多个带有类别名称的列表，例如：-

movies=['spiderman','marvels','thriller']'

sports=['baseball','hockey','football'],

politics=['election','china','usa'] 和许多其他类别。

我只想将 pandas 列 word_list 的关键字与我的类别列表匹配，如果关键字匹配在一起并且如果任何关键字在任何列表中不匹配，则在单独的列中分配相应的列表名称，然后简单地输入miscellaneous 所以，我正在寻找的输出为：-

word_list                                          matched_list_names
['nuclear','election','usa','baseball']            politics,sports,miscellaneous
['football','united','thriller']                   sports,movies,miscellaneous               
['marvels','spiderman','hockey']                   movies,sports

....................                               .....................
....................                               .....................
....................                               ....................

我成功获取到匹配关键字为：-

for i in df['word_list']:
    for j in movies:
        if i in j:
           print (i)

但这给了我匹配的关键字列表。如何获取列表名称并将其添加到 pandas 列？

Answer 1

首先，我认为您应该利用从集合和字典中查找O(1) 的优势。也就是说，我将数据设置为（注意 值是集合）：

d = dict(movies={'spiderman','marvels','thriller'},
         sports={'baseball','hockey','football'},
         politics={'election','china','usa'})

然后，您可以使用您的自定义逻辑transform您的系列

def f(r):
    def m(r_):
        _ = [k for (k, v) in d.items() if r_ in v]
        return _ if _ else ['Misc']
    return {item for z in [m(r_) for r_ in r] for item in z}

df.word_list.transform(f)

0    {Misc, sports, politics}
1      {Misc, sports, movies}
2              {Misc, movies}

对于 300000 行，

%timeit df.word_list.transform(f)
1.1 s ± 22.3 ms per loop (mean ± std. dev. of 7 runs, 1 loop each)

这不是很好但可行..

Answer 2

您可以先展平列表字典，然后通过.get 和miscellaneous 查找不匹配的值，然后转换为sets 以获得唯一类别，并通过join 转换为strings：

movies=['spiderman','marvels','thriller']
sports=['baseball','hockey','football']
politics=['election','china','usa']
d = {'movies':movies, 'sports':sports, 'politics':politics}
d1 = {k: oldk for oldk, oldv in d.items() for k in oldv}

f = lambda x: ','.join(set([d1.get(y, 'miscellaneous') for y in x]))
df['matched_list_names'] = df['word_list'].apply(f)
print (df)

                                 word_list             matched_list_names
0       [nuclear, election, usa, baseball]  politics,miscellaneous,sports
1             [football, united, thriller]    miscellaneous,sports,movies
2  [marvels, hollywood, spiderman, budget]           miscellaneous,movies

列表理解的类似解决方案：

df['matched_list_names'] = [','.join(set([d1.get(y, 'miscellaneous') for y in x])) 
                            for x in df['word_list']]