Spacy 匹配器与跨令牌的正则表达式答案

【问题标题】：Spacy matcher with regex across tokensSpacy 匹配器与跨令牌的正则表达式
【发布时间】：2021-11-05 15:27:01
【问题描述】：

我有以下句子：

phrases = ['children externalize their emotions through outward behavior',
         'children externalize hidden emotions.',
         'children externalize internalized emotions.',
         'a child might externalize a hidden emotion through misbehavior',
         'a kid might externalize some emotions through behavior',
         'traumatized children externalize their hidden trauma through bad behavior.',
         'The kid is externalizing internal traumas',
         'A child might externalize emotions though his outward behavior',
         'The kid externalized a lot of his emotions through misbehavior.']

我想抓住动词externalize之后的任何名词；外化、外化等

在这种情况下；我们应该得到：

externalize their emotions
externalize hidden emotions
externalize internalized emotions
externalize a hidden emotion
externalize some emotions
externalize their hidden trauma
externalizing internal traumas
externalized a lot of his emotions

到目前为止，我只能捕捉到动词 externalize

之后的名词

我想抓住名词；如果它恰好在少于 15 个字符之后。例如：外化很多情绪那应该是匹配的；因为（他的很多）只有14个字符；计算空格。

这是我的作品，远非完美。

import spacy
from spacy.matcher import Matcher
nlp = spacy.load("en_core_web_sm")
matcher =  Matcher(vocab = nlp.vocab)
verb_noun = [{'POS':'VERB'}, {'POS':'NOUN'}]
matcher.add('verb_noun', None, verb_noun)

list_result = []
for phrase in phrases:
    doc = nlp(phrase)
    doc_match = matcher(doc)
    if doc_match:
        for match in doc_match:
            start = match[1]
            end = match[2]
            result = doc[start:end]
            result = [i.lemma_ for i in result]
            if 'externaliz' in result[0].lower():
                result = ' '.join(result)
                list_result.append(result)

【问题讨论】：

假设唯一感兴趣的词是“externalize”、“externalizing”和“externalized”，并且您想返回这些词之一后面的字符串的其余部分和它后面的空格。为此，您可以匹配正则表达式(?:(?<=\bexternalize )|(?<=\bexternalizing )|(?<=\bexternalized )).*。 Demo...
... 问题是将匹配的字符串限制为该行其余部分的一部分（例如，“他的很多情绪”而不是“他的很多情绪通过不当行为引起的”。 )。这将需要自然语言处理，这远远超出了正则表达式的能力。

标签： python nlp spacy matcher

【解决方案1】：

我想抓住名词；如果它恰好在少于 15 个字符之后。比如：外化很多应该匹配的情绪；因为（很多他的）只有14个字符；计算空格。

您可以这样做，但我不建议这样做。您应该做的是编写一个正则表达式来匹配字符串并使用Doc.char_span 创建一个匹配。由于 Matcher 在令牌上工作，因此无法合理地使用诸如“14 个字符，包括空格”之类的启发式方法。此外，这种启发式方法是一种 hack，并且会执行不规律。

我怀疑你真正想做的是弄清楚什么被外化了，也就是说，找到动词的宾语。在这种情况下，您应该使用DependencyMatcher。这是一个使用简单规则并合并名词块的示例：

import spacy

from spacy.matcher import DependencyMatcher
nlp = spacy.load("en_core_web_sm")

texts = ['children externalize their emotions through outward behavior',
         'children externalize hidden emotions.',
         'children externalize internalized emotions.',
         'a child might externalize a hidden emotion through misbehavior',
         'a kid might externalize some emotions through behavior',
         'traumatized children externalize their hidden trauma through bad behavior.',
         'The kid is externalizing internal traumas',
         'A child might externalize emotions though his outward behavior',
         'The kid externalized a lot of his emotions through misbehavior.']

pattern = [
  {
    "RIGHT_ID": "externalize",
    "RIGHT_ATTRS": {"LEMMA": "externalize"}
  },
  {
    "LEFT_ID": "externalize",
    "REL_OP": ">",
    "RIGHT_ID": "object",
    "RIGHT_ATTRS": {"DEP": "dobj"}
  },
]

matcher = DependencyMatcher(nlp.vocab)
matcher.add("EXTERNALIZE", [pattern])

# what was externalized?

# this is optional: merge noun phrases
nlp.add_pipe("merge_noun_chunks")

for doc in nlp.pipe(texts):
    for match_id, tokens in  matcher(doc):
        # tokens[0] is like "externalize"
        print(doc[tokens[1]])

输出：

their emotions
hidden emotions
internalized emotions
a hidden emotion
some emotions
their hidden trauma
internal traumas
emotions
his outward behavior
a lot

【讨论】：