创建具有部分的 Spacy Doc答案

【问题标题】：Create Spacy Doc that has sections创建具有部分的 Spacy Doc
【发布时间】：2019-05-14 02:03:14
【问题描述】：

我想知道当人们想将一个文档分成不同的跨度时，他们为 Spacy 做了什么？例如说我创建了一个文档对象的语料库如下。但对于我正在执行的任务，我想为不同部分创建索引，同时保留原始对象。

doc = nlp("""
Patient History:
    This is paragraph 1.
Assessment:
    This is paragraph 2.
Signature:
    This is paragraph 3.
""")

然后将其解析为：

doc.sections_

会产生

["Patient History", "Assessment", "Signature"]

【问题讨论】：

标签： nlp spacy

【解决方案1】：

这显然必须在文件步骤中出现，并且没有针对管道进行优化，但这是我稍微有点 hacky 的解决方案。

  class ParsedNoteSections(object):
    """
        Pars notes into sections based on entity-tags. All sections are return as newly
        created doc objects.
    """



    def __init__(self,doc):
        self.doc = doc

    def get_section_titles(self):
    """Return the section header titles."""
    return [(e,e.start, e.end) for e in self.doc.ents if e.label_ == 'NOTESECTION']

    def original(self,doc):
        """Retrieve oringal doc object."""
        return self.doc

    def __repr__(self):
        return repr(self.doc)

    def parse_note_sections(self):
        """ Use entity sections as break-points to split original doc.

        Input: 
            None
        Output:
            List of section of objects stored in dictionary.
        """
        section_titles = self.get_section_titles()

        # stopgap for possible errors
        assert len(section_titles) > 0

        doc_section_spans = []
        for idx,section in enumerate(section_titles):
            section_label_new = section[0]
            label_start_new = section[1]
            label_end_new = section[2]

            # store first label
            if idx == 0:
                section_label_old = section_label_new
                continue

            # store last section
            elif idx == 1:
                section_label = section_label_old
                section_doc = self.doc[:label_start_new]

            # if on the last section
            elif idx == len(section_titles) - 1:
                section_label = section_label_old
                section_doc = self.doc[label_start_old:label_start_new]
                doc_section_spans.append({'section_label':section_label, 'section_doc':section_doc})

                section_label = section_label_new
                section_doc = self.doc[label_start_new:]

            # if not storing first or last section
            else:
                section_label = section_label_old
                section_doc = self.doc[label_start_old:label_start_new]

            label_start_old = label_start_new
            section_label_old = section_label_new

            doc_section_spans.append({'section_label':section_label, 'section_doc':section_doc})

        assert len(doc_section_spans) == len(section_titles)

        return doc_section_spans

【讨论】：

【解决方案2】：

SpaCy 不支持“部分”——它们不是文档的通用功能，如何定义它们取决于您处理的是小说、学术论文、报纸、等等

最简单的做法是在将文档送入 spacy 之前自行拆分文档。如果它的格式与您的示例一样，那么使用缩进应该很容易做到。

如果您真的只想拥有一个 Doc 对象，您应该能够使用 spaCy 的管道扩展来管理它。请参阅文档here。

【讨论】：

感谢这也是我解决方案的基本思路。