如何为预训练模型编写评估函数并指定数据加载器？

Question

我是 AllenNLP 库的新手。我正在使用 Pretrained Bidaf-elmo 模型来完成阅读理解任务。我的代码看起来像 -

from allennlp.predictors.predictor import Predictor
import allennlp_models.rc
from allennlp_models import pretrained
from allennlp.training.util import evaluate
import allennlp.data.data_loaders.simple_data_loader

archive_file_path = "https://storage.googleapis.com/allennlp-public-models/bidaf-model-2020.03.19.tar.gz"
input_path = "C:\\Users\\SHRIPRIYA\\sample_dataset.json"

data_load = simple_data_loader(input_path)
evaluate(model=archive_file_path, data_loader = data_load, output_file=output, predictions_output_file=pred_output_file, cuda_device=0)

simple_data_loader() 行引发错误 - name 'simple_data_loader' is not defined。我知道这是一个语法错误，但我找不到任何示例来使用 AllenNLP 的数据加载器函数加载 JSON 文件并使用预训练模型对其进行评估。

关于我的数据：

1. 总样本通道 = 10,000
2. 问题总数 = 1000

每个示例段落都需要回答所有 1000 个问题。我的示例 JSON 输入看起来像 -

{
  "passage": "Venus is named after the Roman goddess of love and beauty. Venus is the second planet from the sun. Is the brightest object in the sky besides our Sun and the Moon. Venus has no moons. It is also known as the morning star because at sunrise it appears in the east. It is also known as the evening star as it appears at sunset when it is in the west. It cannot be seen in the middle of the night. Venus and Earth are close together in space and similar in size, which is the reason Venus is called Earth's sister planet. Venus has more volcanoes than any other planet. It is the hottest planet in the solar system, even hotter than Mercury, which is closer to the Sun. The temperature on the surface of Venus is about 460° Celsius. The atmosphere on Venus is composed of carbon dioxide. The surface is heated by radiation from the sun, but the heat cannot escape through the clouds and layer of carbon dioxide. (This is a “greenhouse effect”).",
  "questions": [
    "How many moons does Venus have?",
    "Venus was named after which Roman goddess?",    
    "At what position does Venus lie from Sun?",
    "What is the temperature of Venus surface?",
    "Why is Venus called Earth’s sister planet?",
    "What is the atmosphere of Venus composed of?"
  ]}

如果有任何更快的替代方法来评估多个段落的多个问题，请告诉我。

谢谢！

Answer 1

从您的代码看来，您没有正确导入SimpleDataLoader。以下应该有效： from allennlp.data.data_loaders import SimpleDataLoader

您可以在此处找到训练和评估模型的示例：https://guide.allennlp.org/training-and-prediction#3