查找从一个 csv 列到另一个 csv 列的单词频率

Question

我是 python 新手，我有两个 csv 文件，一个包含细菌名称

import csv
import pandas as pd

from collections import Counter
import re
import operator

#Bacteria File Open

Bac = []
with open ("/home/shayez/Desktop/Bacteria.csv", "r") as csv_file1:
    csv_reader1 = csv.reader(csv_file1,delimiter = ',')

    for lines1 in csv_reader1:
        Bac.append(lines1)
       # print(lines1[0])


#Abstract File Open
Abs = []
with open ("/home/shayez/Desktop/Anti.csv", "r") as csv_file:
    csv_reader = csv.reader(csv_file,delimiter = ',')

    for lines in csv_reader:
        Abs.append(lines[2])



abswordlist = []
 for ab in Abs:
 abswordlist.append(Counter(ab.split()))

 #print (abswordlist)

 cntword = Counter(Abs)

 for Bac in Bac:
 print (f"{Bac}:{abswordlist[Bac]}")

像这样：-

这是包含大约 2200 个细菌名称的细菌文件

包含摘要的第二个文件 像这样 ：-

我必须将第一个细菌文件名的单词与第二个摘要列进行比较，并将细菌的频率计数到摘要中并保存到第三个 csv

像这样：-

Answer 1

我建议您使用 pandas 库来执行此任务，因为您似乎需要做很多聚合。

由于您不提供 [mcve]，我必须自己制作一个。 因此，您必须阅读您的第一个 csv 并将值保存为列表。它们稍后将成为您保留的专栏。

然后...使用这个数组。我建议你使用.apply()，结合split() 和Counter()（来自python 集合）。 然后，join() 所有这一切都使用json_normalize()。

import pandas as pd

from collections import Counter
from pandas.io.json import json_normalize

to_keep = ['LONER', 'I', 'AM']

df = pd.DataFrame({
        'date' : ['some date', 'some_other_date', 'lol date'],
        'garbage' : ['I AM A LONER', 'AND SO AM I LOL', 'some other weird sentence']
    })
print(df.head())
#               date                    garbage
# 0        some date               I AM A LONER
# 1  some_other_date            AND SO AM I LOL
# 2         lol date  some other weird sentence

# Here I am showing you the inside of what I insert into json_normalize.
# It basically counts the word occurrences per line. You split the words,    
# and count the list items using `Counter()`
print(df['garbage'].apply(lambda x:Counter(x.split())))
# 0                {'I': 1, 'AM': 1, 'A': 1, 'LONER': 1}
# 1       {'AND': 1, 'SO': 1, 'AM': 1, 'I': 1, 'LOL': 1}
# 2    {'some': 1, 'other': 1, 'weird': 1, 'sentence'...

# Then, you use the json_normalize() function to turn all your jsons into a big DataFrame. And join the result to the previously created DataFrame.
df = df.join( json_normalize(df['garbage'].apply(lambda x:Counter(x.split()))) )
print(df)
#               date                    garbage    A  ...    sentence  some  weird
# 0        some date               I AM A LONER  1.0  ...         NaN   NaN    NaN
# 1  some_other_date            AND SO AM I LOL  NaN  ...         NaN   NaN    NaN
# 2         lol date  some other weird sentence  NaN  ...         1.0   1.0    1.0

# And keep the first indices, here, only date, in addition of the columns you wished to keep earlier.
final_df = df[ ['date'] + [*to_keep] ]
print(final_df)
#               date  LONER    I   AM
# 0        some date    1.0  1.0  1.0
# 1  some_other_date    NaN  1.0  1.0
# 2         lol date    NaN  NaN  NaN