spaCy 仅在调用“rasa train”时无法加载模型答案

【问题标题】：spaCy can't load model ONLY when calling "rasa train"spaCy 仅在调用“rasa train”时无法加载模型
【发布时间】：2020-02-06 05:45:33
【问题描述】：

我正在通过命令行训练一个 rasa 模型，但 spaCy 似乎无法加载我的语言模型 pt_core_news_sm 只有当我尝试通过终端进行训练时。

一切都在我的 venv 中完成并以管理员身份执行；

A 在从 python 调用 spaCy 时可能会加载模型：

import spacy
spacy.load("pt_core_news_sm")

在 python 脚本中通过rasa_nlu 进行训练也可以。

但是当我打电话给rasa train 时，我得到了这个回溯：

2020-02-06 02:03:19 INFO     rasa.nlu.utils.spacy_utils  - Trying to load spacy model with name 'pt_core_news_sm'
Traceback (most recent call last):
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\utils\spacy_utils.py", line 51, in load_model
    return spacy.load(spacy_model_name, disable=["parser"])
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\spacy\__init__.py", line 30, in load
    return util.load_model(name, **overrides)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\spacy\util.py", line 169, in load_model
    raise IOError(Errors.E050.format(name=name))
OSError: [E050] Can't find model 'pt_core_news_sm'. It doesn't seem to be a shortcut link, a Python package or a valid path to a data directory.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\runpy.py", line 193, in _run_module_as_main
    "__main__", mod_spec)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\runpy.py", line 85, in _run_code
    exec(code, run_globals)
  File "C:\Users\aliss\AppData\Local\Programs\Python\Python36\Scripts\rasa.exe\__main__.py", line 9, in <module>
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\__main__.py", line 76, in main
    cmdline_arguments.func(cmdline_arguments)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\cli\train.py", line 76, in train
    additional_arguments=extract_additional_arguments(args),
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 50, in train
    additional_arguments=additional_arguments,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\asyncio\base_events.py", line 473, in run_until_complete
    return future.result()
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 101, in train_async
    additional_arguments,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 188, in _train_async_internal
    additional_arguments=additional_arguments,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 245, in _do_training
    persist_nlu_training_data=persist_nlu_training_data,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\train.py", line 474, in _train_nlu_with_validated_data
    persist_nlu_training_data=persist_nlu_training_data,
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\train.py", line 74, in train
    trainer = Trainer(nlu_config, component_builder)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\model.py", line 147, in __init__
    self.pipeline = self._build_pipeline(cfg, component_builder)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\model.py", line 159, in _build_pipeline
    component = component_builder.create_component(component_cfg, cfg)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\components.py", line 515, in create_component
    component = registry.create_component_by_config(component_config, cfg)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\registry.py", line 228, in create_component_by_config
    return component_class.create(component_config, config)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\utils\spacy_utils.py", line 81, in create
    nlp = cls.load_model(spacy_model_name)
  File "c:\users\aliss\appdata\local\programs\python\python36\lib\site-packages\rasa\nlu\utils\spacy_utils.py", line 58, in load_model
    "en_core_web_md en".format(spacy_model_name)
rasa.nlu.model.InvalidModelError: Model 'pt_core_news_sm' is not a linked spaCy model.  Please download and/or link a spaCy model, e.g. by running:
python -m spacy download en_core_web_md
python -m spacy link en_core_web_md en

我用python和python3尝试了以下下载策略。第一个似乎是this question 的答案，但效果不佳：

pip install https://github.com/explosion/spacy-models/releases/download/pt_core_news_sm-2.2.5/pt_core_news_sm-2.2.5.tar.gz

python -m spacy download pt

python -m spacy download pt_core_news_sm

这是我的配置文件：

language: "pt_core_news_sm"

pipeline:
- name: "SpacyNLP"                 
- name: "tokenizer_spacy"          
- name: "ner_crf"                
- name: "intent_featurizer_spacy"     
- name: "intent_classifier_sklearn"  
- name: "ner_synonyms"               

policies:
  - name: "FormPolicy"

我正在使用...

spacy==2.2.3

rasa==1.7.0
rasa-core==0.14.5
rasa-core-sdk==0.14.0
rasa-nlu==0.15.0
rasa-sdk==1.7.0

【问题讨论】：

你确定你是在同一个 python 环境中尝试这个吗？另外，您最近是否更新了 spacy 版本，或者您正在使用几种不同的环境？
是的，我已经更新了，但是，正如我所说，通过 rasa_nlu 在 python 中训练效果很好。我发现，在命令行训练中，rasa 会在我的 virtualenv 之外搜索语言模型，即使我从内部调用它也是如此。在虚拟环境之外下载我的语言模型解决了它。

标签： spacy language-model rasa

【解决方案1】：

解决了。显然，从 venv 内部调用 rasa train 会搜索我的 virtualenv 之外的语言模型。我必须下载在 venv 之外执行 python 的模型，它完成了训练。即便如此，我也不知道为什么会这样。

【讨论】：

【解决方案2】：

对于 Windows 用户 --

请先检查您的 Python 环境。设置 --> 项目解释器。如果是 Anaconda 虚拟环境。

以管理员身份打开CMD（右键cmd，选择以管理员身份运行）

激活您的环境。例如：

激活 rasaTesting（您的项目/环境的名称）

为 spacy 模型创建快捷链接，例如（如果还没有，请下载 spacy 模型）-

python -m spacy link en_core_web_md en

附上环境的图片。

【讨论】：