运行零镜头模型时错误说明缺少标记器答案

【问题标题】：Error stating missing tokenizer when running zero shot model运行零镜头模型时错误说明缺少标记器
【发布时间】：2020-06-05 02:11:20
【问题描述】：

此示例取自拥抱脸，但会产生模型加载错误。我以为我下载了这个模型，但是否知道我可以如何避免这个错误？谢谢，威尔

from ktrain import text 
zsl = text.ZeroShotClassifier()
topic_strings=['politics', 'elections', 'sports', 'films', 'television']
doc = 'I am extremely dissatisfied with the President and will definitely vote in 2020.'
zsl.predict(doc, topic_strings=topic_strings, include_labels=True)

结果：

OSError：在标记器模型名称列表（bart-large、bart-large-mnli、bart-large-cnn、bart-large-xsum）中找不到模型名称“facebook/bart-large-mnli”。我们假设 'facebook/bart-large-mnli' 是路径、模型标识符或指向包含名为 ['vocab.json', 'merges.txt'] 的词汇文件的目录的 url，但在以下位置找不到此类词汇文件此路径或网址。

【问题讨论】：

标签： deep-learning nlp

【解决方案1】：

derp 我手头没有默认模型，所以我将第 2 行更改为指向我能够正确加载的模型：

zsl = text.ZeroShotClassifier(model_name='bart-large-mnli')

【讨论】：

【解决方案2】：

您可能正在使用早于 2.11 的 transformers 版本。从 2.11 transformers 开始，必须使用完整的模型 ID 指定 BART（和一些其他模型），如 transformers 2.11 的 CHANGELOG 中所示：

URLs to model weights are not hardcoded anymore (@julien-c)
Archive maps were dictionaries linking pre-trained models to their S3 URLs. Since the arrival of the model hub, these have become obsolete.

⚠️ This PR is breaking for the following models: BART, Flaubert, bert-japanese, bert-base-finnish, bert-base-dutch. ⚠️
Those models now have to be instantiated with their full model id:

"cl-tohoku/bert-base-japanese"
"cl-tohoku/bert-base-japanese-whole-word-masking"
"cl-tohoku/bert-base-japanese-char"
"cl-tohoku/bert-base-japanese-char-whole-word-masking"
"TurkuNLP/bert-base-finnish-cased-v1"
"TurkuNLP/bert-base-finnish-uncased-v1"
"wietsedv/bert-base-dutch-cased"
"flaubert/flaubert_small_cased"
"flaubert/flaubert_base_uncased"
"flaubert/flaubert_base_cased"
"flaubert/flaubert_large_cased"

all variants of "facebook/bart"

【讨论】：