【问题标题】：How to extract single word (not larger word containing it) in pandas dataframe?如何在熊猫数据框中提取单个单词（不是包含它的更大单词）？
【发布时间】：2019-12-31 18:48:33
【问题描述】：

我想这样提取单词：

a dog ==> dog
some dogs ==> dog
dogmatic ==> None

有一个类似的链接： Extract substring from text in a pandas DataFrame as new column

但它不符合我的要求。

从此数据框：

df = pd.DataFrame({'comment': ['A likes cat', 'B likes Cats',
                               'C likes cats.', 'D likes cat!', 
                               'E is educated',
                              'F is catholic',
                              'G likes cat, he has three of them.',
                              'H likes cat; he has four of them.',
                              'I adore !!cats!!',
                              'x is dogmatic',
                              'x is eating hotdogs.',
                              'x likes dogs, he has three of them.',
                              'x likes dogs; he has four of them.',
                              'x adores **dogs**'
                              ]})

如何得到正确的输出？

                            comment      label EXTRACT
0                           A likes cat   cat     cat
1                          B likes Cats   cat     cat
2                         C likes cats.   cat     cat
3                          D likes cat!   cat     cat
4                         E is educated  None     cat
5                         F is catholic  None     cat
6    G likes cat, he has three of them.   cat     cat
7     H likes cat; he has four of them.   cat     cat
8                      I adore !!cats!!   cat     cat
9                         x is dogmatic  None     dog
10                 x is eating hotdogs.  None     dog
11  x likes dogs, he has three of them.   dog     dog
12   x likes dogs; he has four of them.   dog     dog
13                    x adores **dogs**   dog     dog

注意：列 EXTRACT 给出错误答案，我需要列标签。

【问题讨论】：

那么第1行不应该也返回None吗？是Cats 不是Cat
@anky_91，抱歉混淆，猫或猫与猫（动物猫）标签相同。

标签： python regex pandas

【解决方案1】：

我们可以将str.extract 与negative lookahead 一起使用：?!。我们检查匹配后的字符是否不超过 2 个字母。例如dogmatic:

之后我们使用np.where 和positive lookahead。伪逻辑如下：

所有“dog”或“cat”前面带字母字符的行都将被替换为NaN

words = ['cat', 'dog']

df['label'] = df['comment'].str.extract('(?i)'+'('+'|'.join(words)+')(?![A-Za-z]{2,})')
df['label'] = np.where(df['comment'].str.contains('(?<=\wdog)|(?<=\wcat)'), np.NaN, df['label'])

输出

                                comment label
0                           A likes cat   cat
1                          B likes Cats   Cat
2                         C likes cats.   cat
3                          D likes cat!   cat
4                         E is educated   NaN
5                         F is catholic   NaN
6    G likes cat, he has three of them.   cat
7     H likes cat; he has four of them.   cat
8                      I adore !!cats!!   cat
9                         x is dogmatic   NaN
10                 x is eating hotdogs.   NaN
11  x likes dogs, he has three of them.   dog
12   x likes dogs; he has four of them.   dog
13                    x adores **dogs**   dog

【讨论】：

hotdogs 应该会失败。
所以我应该投反对票吗？与我在这里找到的许多其他问题不同，OP 提供了一组很好的测试数据。如果您未能匹配未指定的边缘情况，这是可以理解的，但您至少可以做的是匹配 OP 呈现的内容。
你去@MonkeyZeus
@Erfan 我真的很感谢你的意见和支持，但我仍然有大约百万行，hotdog 只是边缘情况的一个例子。有没有办法去掉这些包含小词的大词？
很好，现在我很乐意支持这个答案。

【解决方案2】：

df = pd.DataFrame({'comment': ['A likes cat', 'B likes Cats',
                           'C likes cats.', 'D likes cat!', 
                           'E is educated',
                          'F is catholic',
                          'G likes cat, he has three of them.',
                          'H likes cat; he has four of them.',
                          'I adore !!cats!!',
                          'x is dogmatic',
                          'x is eating hotdogs.',
                          'x likes dogs, he has three of them.',
                          'x likes dogs; he has four of them.',
                          'x adores **dogs**'
                          ]})

word_list = ["cat", "cats", "dog", "dogs"]    # words (and all variations) that you wish to check for

df["label"] = df["comment"].str.lower().str.replace('[^\w\s]','').str.split().apply(lambda x: [i for i in word_list if i in x])
df["label"] = df["label"].apply(lambda x: None if not x else x)
df["label"] = df["label"].str.replace("[","").str.replace("]","").str.replace("'","").str.replace("s","")

那么这给了你：

df
    comment                             label
0   A likes cat                         cat
1   B likes Cats                        cat
2   C likes cats.                       cat
3   D likes cat!                        cat
4   E is educated                       None
5   F is catholic                       None
6   G likes cat, he has three of them.  cat
7   H likes cat; he has four of them.   cat
8   I adore !!cats!!                    cat
9   x is dogmatic                       None
10  x is eating hotdogs.                None
11  x likes dogs, he has three of them. dog
12  x likes dogs; he has four of them.  dog
13  x adores **dogs**                   dog

【讨论】：

我喜欢你的答案，它简单直观，被选为主要答案。非常感谢。
也改了df["label"].apply(lambda x: None if not x else x[0]) 然后就不用去掉括号了。
@MilkyWay001 很高兴听到，很乐意提供帮助。是的，这是个好主意。

【解决方案3】：

您想要实现的是提取句子的标签。这是自然语言处理问题而不是编程问题。

方法：

使用 stemmer/lemmatizer 。您可以将词干分析器的输出与词干分类名称列表相匹配。这很可能无法为您提供足够高的准确度。
根据您的主题/标签训练机器学习分类器。

Lemmatizer 解决方案 - 我使用了来自 another answer in this question 的一些预处理代码

import nltk
import pandas as pd

lemma = nltk.wordnet.WordNetLemmatizer()
nltk.download('wordnet')


df = pd.DataFrame({'comment': ['A likes cat', 'B likes Cats',
                           'C likes cats.', 'D likes cat!', 
                           'E is educated',
                          'F is catholic',
                          'G likes cat, he has three of them.',
                          'H likes cat; he has four of them.',
                          'I adore !!cats!!',
                          'x is dogmatic',
                          'x is eating hotdogs.',
                          'x likes dogs, he has three of them.',
                          'x likes dogs; he has four of them.',
                          'x adores **dogs**'
                          ]})

word_list = ["cat",  "dog"]    # words (and all variations) that you wish to check for
word_list = list(map(lemma.lemmatize, word_list))


df["label"] = df["comment"].str.lower().str.replace('[^a-zA-Z]', ' ').apply(lambda x: [ lemma.lemmatize(word) for word in x.split()  ])
df["label"] = df["label"].apply(lambda x: [i for i in word_list if i in x])

df["label"] = df["label"].apply(lambda x: None if not x else x)
print(df)

【讨论】：

感谢您的建议。实际上，我将在获得标签后构建分类器。但首先我需要拿到标签。你有任何链接如何从这样的句子创建标签？
你能在 Twitter API 中挖掘你需要的带有哈希标签的句子吗？
您需要进行网络搜索。我不知道为您的特定分类问题生成标记数据的所有可能性。很抱歉没有提供更多帮助。
同意这个答案。除非您对要做什么有更具体的描述，否则问题似乎太笼统了。即使处理英语的复数也不是那么容易（dwarf -> dwarves, fox -> foxes, but not did -> doe）
那么你的问题应该指定你想识别“狗、狗、猫、猫，但没有别的”。这是一个更清晰的问题，也更容易回答。

【解决方案4】：

这样的？

/^(.*?[^a-z\r\n])?((cat|dog)s?)([^a-z\r\n].*?)?$/gmi

\2 将包含以下之一：猫、狗、猫、狗

https://regex101.com/r/Tt3MiZ/3

【讨论】：

df.comment.str.extract(regexp, expand=False) 给出所有的 NaN。如何在 pandas 中实现这一点？
@MilkyWay007 老实说，我不知道，我从来没有做过 Python。在 regex101 链接中，您可以单击“代码生成器”以获取该站点如何实现其结果的示例，以便您可以使其符合您的需求。我相信有很多关于 SO 的 Pandas 问题供您参考。

【解决方案5】：

在这种情况下，我想您甚至不需要使用正则表达式。只需使用等于运算符 == 来指定精确匹配，因为您正在寻找 "dog" "dogs" "cat" "cats" 作为整个单词。例如：

for word in string:
    if word == "dogs":
        print("Yes")
    else:
        print("No")

如果你的字符串是“他喜欢热狗”，上面的循环将返回“否”

【讨论】：

【解决方案6】：

您可以为 cat、cats、dog 和 dogs 编译正则表达式。

import re
regex = re.compile(r'cats', re.I)
data = ['A likes cat', 'B likes Cats',
                           'C likes cats.', 'D likes cat!', 
                           'E is educated',
                          'F is catholic',
                          'G likes cat, he has three of them.',
                          'H likes cat; he has four of them.',
                          'I adore !!cats!!',
                          'x is dogmatic',
                          'x is eating hotdogs.',
                          'x likes dogs, he has three of them.',
                          'x likes dogs; he has four of them.',
                          'x adores **dogs**'
                          ]
for i in data:
    t = regex.search(i)
    print(t)

【讨论】：