使用 pandas 根据其他数据 csv 检查数据

Question

我有两个数据 csv 第一个：

word,centroid
she,1
great,0
good,3
mother,2
father,2
After,4
before,4
.....

第二个：

sentences,label
good mother,1
great father,1

我想根据聚类结果检查每个句子 因此，如果centroid 3 上的句子是good mother good，则数组将为[0,0,0,1,0]，centroid 2 上的单词mother 则数组将为[0, 0,1,1,0]...

我有复杂而错误的代码...谁能帮帮我

这是我的代码：

import pandas as pd
import re
array=[]
data = pd.read_csv('data/data_komentar.csv',encoding = "ISO-8859-1") 
df = pd.read_csv('data/hasil_cluster.csv',encoding = "ISO-8859-1") 
for index,row in data.iterrows():
    kalimat=row[0]
    words=re.sub(r'([^\s\w]|_)', '', str(kalimat))
    words= re.sub(r'[0-9]+', '', words)
    for word in words.split():    
        kata=word.lower()
        df = df[df.eq(kata)]
        if df.empty:
            print("empty")
        else:
            print(kata)
            if df['centroid;'] is 0:
                array=array+[1,0,0,0,0]
            if df['centroid'] is 1:
                array=array+[0,1,0,0,0]
            if df['centroid'] is 2:
                array=array+[0,0,1,0,0]
            if df['centroid;'] is 3:
                array=array+[0,0,0,1,0]
            if df['centroid;'] is 4:
                array=array+[0,0,0,0,1]
            print(array)

Answer 1

您可以在 DataFrame 的sentences 列上使用apply()：

import numpy as np

MAX_CENTROIDS = 5

def get_centroids(row):
    centroids = np.zeros(MAX_CENTROIDS, dtype=int)
    for word in row.split(' '):
        if word in df1['word'].values:
            centroids[df1[df1['word']==word]['centroid'].values]+=1
    return centroids

df2['centroid'] = df2['sentences'].apply(get_centroids)

结果df2:

df1 是带有您的单词和质心的 DataFrame，df2 带有句子。您必须在MAX_CENTROIDS 中指定质心的最大数量（=质心列表的长度）。

编辑

要阅读您提供的数据样本：

# Maybe remove encoding on your system
df1 = pd.read_csv('hasil_cluster.csv', sep=',', encoding='iso-8859-1')

# Drop Values without a centroid:
df1.dropna(inplace=True)

# Remove ; from every centroid value and convert the column to integers
df1['centroid'] = df1['centroid;'].apply(lambda x:str(x).replace(';', '')).astype(int)

# Remove unused colum
df1.drop('centroid;', inplace=True, axis=1)