从文本中提取引文和归属答案

【问题标题】：Extract quotations and attribution from text从文本中提取引文和归属
【发布时间】：2022-12-19 17:03:55
【问题描述】：

我试图从文本中提取引文和引文属性（即说话者），但我没有获得所需的输出。我正在使用文本。到目前为止，这是我尝试过的：

import textacy
from textacy import extract
from textacy.representations import Vectorizer

data = [
        ("\"Hello, nice to meet you,\" said world 1", {"url": "example1.com", "date": "Jan 1"}),
        ("\"Hello, nice to meet you,\" said world 2", {"url": "example2.com", "date": "Jan 2"}),
        ]

corpus = textacy.Corpus("en_core_web_sm", data=data)

vectorizer = Vectorizer(tf_type="linear", idf_type="smooth")
doc = vectorizer.fit_transform(
    ((term.lemma_ for term in extract.terms(doc, ngs=1, ents=True)) for doc in corpus)
    ) 
         
quotes = (textacy.extract.triples.direct_quotations(doc) for records in doc)

print(list(quotes))

这是输出：

[<generator object direct_quotations at 0x7fdc0faaf6d0>, <generator object direct_quotations at 0x7fdc0faaf5f0>]

所需的输出是这样的：
[DQTriple(speaker=[world 1], cue=[said], content="你好，很高兴认识你，")] [DQTriple(speaker=[world 2], cue=[said], content="你好，很高兴认识你，")]

编辑

这是一些改进的代码，现在使用语料库而不是数据创建文档：
import textacy
from textacy import extract
from textacy.representations import Vectorizer

data = [
        ("\"Hello, nice to meet you,\" said world 1", {"url": "example1.com", "date": "Jan 1"}),
        ("\"Hello, nice to meet you,\" said world 2", {"url": "example2.com", "date": "Jan 2"}),
        ]

corpus = textacy.Corpus("en_core_web_sm", data=data)

vectorizer = Vectorizer(tf_type="linear", idf_type="smooth")
doc = vectorizer.fit_transform(
    ((term.lemma_ for term in extract.terms(corpus, ngs=1, ents=True)) for record in corpus)
    ) 
         
print(list((textacy.extract.triples.direct_quotations(doc))))
但是现在我有一个新的错误：

AttributeError: 'spacy.tokens.doc.Doc' 对象没有属性 'is_space'

【问题讨论】：

您是否尝试将生成器变成列表 (list(textacy.extract.triples.direct_quotations(doc)))？
刚刚尝试过，这可能是正确的方法，但后来我收到错误 raise AttributeError(attr + " not found") AttributeError: lang_ not found，这也是我通过其他方法得到的错误。所以我认为还有其他问题。
看here和here。您必须安装 spaCy 语言特定的模型数据来解决这个问题：python -m spacy download en_core_web_sm
我实际上已经安装了它，只是为了确定才重新安装。还安装了python -m textacy download lang_identifier --version 2.0。仍然有同样的错误。

标签： python function nlp text-extraction textacy

【解决方案1】：

这有效：

data = [
        (""Hello, nice to meet you," said world 1"),
        (""Hello, nice to meet you," said world 2"),
        ]
for record in data:
    doc = textacy.make_spacy_doc(record, lang="en_core_web_sm")
    print(list(textacy.extract.triples.direct_quotations(doc)))

_{此答案由 OP jedmund 在 CC BY-SA 4.0 下发布为 edit 问题 Extract quotations and attribution from text。}

【讨论】：