使用 nltk 停用词从 pandas 列中的列表中删除停用词时，删除停用词失败答案

【问题标题】：removing stopwords fails when using nltk stopwords to remove them from a list in a pandas column使用 nltk 停用词从 pandas 列中的列表中删除停用词时，删除停用词失败
【发布时间】：2021-02-02 18:57:06
【问题描述】：

我有一个带有字符串条目的数据框，我正在使用一个函数来删除停用词。单元格编译但未产生预期结果。

df['column'].iloc[0] = 'BK HE HAS KITCHEN TROUBLE WITH HIS BLENDER'

def text_process(text):
    try :
        nopunc = [char for char in text if char not in sting.punctuation]
        nopunc = ' '.join(nopunc)
        return [word for word in nopunc.split() if word.lower not in stopwords.words('english')
    except TypeError: return []

df['column'].apply(text_process)

The first cell results look like this : 
['BK ', 'HE', 'HAS', 'KITCHEN', 'TROUBLE', 'WITH', 'HIS', 'BLENDER']

(He, has, with, his) 应该被删除，但它们仍然出现在单元格中？谁能解释这是如何发生的或如何解决它？

【问题讨论】：

标签： python nltk stop-words

【解决方案1】：

from nltk.corpus import stopwords 
from nltk.tokenize import word_tokenize 

example_sent = "BK HE HAS KITCHEN TROUBLE WITH HIS BLENDER"
example_sent=example_sent.lower()
stop_words = set(stopwords.words('english')) 

word_tokens = word_tokenize(example_sent) 

filtered_sentence = [w for w in word_tokens if not w in stop_words] 

filtered_sentence = [] 

for w in word_tokens: 
    if w not in stop_words: 
       filtered_sentence.append(w) 


print(word_tokens) 
print(filtered_sentence)

['bk', 'he', 'has', 'kitchen', 'trouble', 'with', 'his', 'blender']

['bk', '厨房', '麻烦', '搅拌机']

【讨论】：