你如何在 Spacy 中使用自定义的 NeuralCoref 模型？答案

【问题标题】：How do you use a custom NeuralCoref model in Spacy?你如何在 Spacy 中使用自定义的 NeuralCoref 模型？
【发布时间】：2020-02-15 21:33:40
【问题描述】：

我遵循了有关如何使用 NeuralCoref 进行train a a neural coreference 建模的指南。我现在有一个模型，但不知道如何在 Spacy 中使用 coref 模型。

手册中的以下内容并未描述如何加载自定义模型：

# Load your usual SpaCy model (one of SpaCy English models)
import spacy
nlp = spacy.load('custom-danish-spacy-model')

# Add neural coref to SpaCy's pipe
import neuralcoref
neuralcoref.add_to_pipe(nlp)

# You're done. You can now use NeuralCoref as you usually manipulate a SpaCy document annotations.
doc = nlp(u'A sentence in Danish. Another sentence in the same language.')

编辑：我尝试将经过训练的模型（通过运行 python -m neuralcoref.train.learn --train ./data/train/ --eval ./data/dev/ 生成）放入 NeuralCoref 缓存文件夹并运行上面的代码。出现以下错误：

  return f(*args, **kwds)
/home/johan/Code/spacy-neuralcoref/venv/lib/python3.6/importlib/_bootstrap.py:219: RuntimeWarning: spacy.vocab.Vocab size changed, may indicate binary incompatibility. Expected 96 from C header, got 104 from PyObject
  return f(*args, **kwds)
Traceback (most recent call last):
  File "custom_model_test.py", line 5, in <module>
    neuralcoref.add_to_pipe(nlp)
  File "/home/johan/Code/spacy-neuralcoref/neuralcoref/neuralcoref/__init__.py", line 42, in add_to_pipe
    coref = NeuralCoref(nlp.vocab, **kwargs)
  File "neuralcoref.pyx", line 554, in neuralcoref.neuralcoref.NeuralCoref.__init__
  File "neuralcoref.pyx", line 947, in neuralcoref.neuralcoref.NeuralCoref.from_disk
  File "/home/johan/Code/spacy-neuralcoref/venv/lib/python3.6/site-packages/thinc/neural/_classes/model.py", line 355, in from_bytes
    data = srsly.msgpack_loads(bytes_data)
  File "/home/johan/Code/spacy-neuralcoref/venv/lib/python3.6/site-packages/srsly/_msgpack_api.py", line 29, in msgpack_loads
    msg = msgpack.loads(data, raw=False, use_list=use_list)
  File "/home/johan/Code/spacy-neuralcoref/venv/lib/python3.6/site-packages/srsly/msgpack/__init__.py", line 60, in unpackb
    return _unpackb(packed, **kwargs)
  File "_unpacker.pyx", line 199, in srsly.msgpack._unpacker.unpackb
srsly.msgpack.exceptions.ExtraData: unpack(b) received extra data.

【问题讨论】：

标签： python spacy msgpack

【解决方案1】：

您的问题更多是 msgpack 的使用而不是 SpaCy。我建议您查看How to import text from CoNNL format with named entities into spaCy, infer entities with my model and write them to the same dataset (with Python)? 以更清楚地了解问题的根源。

在您所拥有的情况下，模型存储代码和加载代码之间的格式很可能不兼容，但没有用于训练/存储模型的确切版本的 SpaCy 和 NeuralCoref 代码，很难准确判断.

【讨论】：

【解决方案2】：

你可以通过将spacy的包降级到2.0.13重新运行，先安装cymem==1.31.2再安装spacy==2.0.12，安装neural coref的时候一定要记住顺序。

!pip install https://github.com/huggingface/neuralcoref-models/releases/download/en_coref_md-3.0.0/en_coref_md-3.0.0.tar.gz !pip install cymem==1.31.2 spacy==2.0.12

【讨论】：