pandas DataFrame 多个子字符串匹配，也将一行的特定匹配子字符串放入新列答案

【问题标题】：pandas DataFrame multiple substrings match, also put the particular matched substring for a row into a new columnpandas DataFrame 多个子字符串匹配，也将一行的特定匹配子字符串放入新列
【发布时间】：2018-11-27 16:36:50
【问题描述】：

我正在尝试从调查响应 DF 中提取一些记录。所有这些记录都需要至少包含一些关键词之一。例如：现在我有一个数据框 df:

svy_rspns_txt
I like it
I hate it
It's a scam
It's shaddy
Scam!
Good service
Very disappointed

现在如果我跑

kw="hate,scam,shaddy,disappoint"
sensitive_words=[unicode(x,'unicode-escape') for x in kw.lower().split(",")]
df=df[df["svy_rspns_txt"].astype('unicode').str.contains('|'.join(sensitive_words),case=False,na=False)]

我会得到类似的结果

svy_rspns_txt
I hate it
It's a scam
It's shaddy
Scam!
Very disappointed

现在如何添加“matched_word”列来显示匹配的确切字符串，以便获得如下结果：

svy_rspns_txt            matched_word
I hate it                hate
It's a scam              scam
It's shaddy              shaddy
Scam!                    scam
Very disappointed        disappoint

【问题讨论】：

如果多个单词匹配怎么办？ matched_word 列应该显示所有单词，还是只显示匹配的第一个单词？
@TimJohns 现在我只需要出现第一个单词。但是，如果您能提供一些建议来显示所有匹配的单词，那就太好了。谢谢！

标签： python pandas substring match string-matching

【解决方案1】：

使用带有next的生成器表达式：

df = pd.DataFrame({'text': ["I like it", "I hate it", "It's a scam", "It's shaddy",
                            "Scam!", "Good service", "Very disappointed"]})

kw = "hate,scam,shaddy,disappoint"

words = set(kw.split(','))

df['match'] = df['text'].apply(lambda x: next((i for i in words if i in x.lower()), np.nan))

print(df)

                text       match
0          I like it         NaN
1          I hate it        hate
2        It's a scam        scam
3        It's shaddy      shaddy
4              Scam!        scam
5       Good service         NaN
6  Very disappointed  disappoint

您可以通过pd.Series.notnull 或注释NaN != NaN 过滤有效字符串：

res = df[df['match'].notnull()]
# or, res = df[df['match'].notna()]
# or, res = df[df['match'] == df['match']]

print(res)

                text       match
1          I hate it        hate
2        It's a scam        scam
3        It's shaddy      shaddy
4              Scam!        scam
6  Very disappointed  disappoint

【讨论】：

你不能只做res = df[df['match'].notna()]，让它更清楚吗？
@TimJohns，是的，这也有效，而且更清晰，谢谢。我通常使用pd.Series.notnull。