为什么运行GPT2小模型时找不到encoder.json

Question

晚上好，

请注意，我不是 Python 或机器学习专家

我正在尝试运行 GPT2 的小实例，在大肆宣传之后我想检查一下。到目前为止，我已经下载了所有先决条件。 Python、正则表达式、张量流等，但是在运行脚本以从模型中生成样本时，我会抛出以下错误

'''文件 "C:*****\F******y\Desktop\Python\gpt-2\src\encoder.py"，第 109 行，在 get_encoder 使用 open(os.path.join(models_dir, model_name, 'encoder.json'), 'r') 作为 f： FileNotFoundError: [Errno 2] 没有这样的文件或目录：'models\124M\encoder.json'''

当我调用脚本时，我切换到保存文件的目录并从命令行运行 ''' generate_unconditional_samples.py --top_k 40 '''

脚本本身是这样的

#!/usr/bin/env python3

import fire
import json
import os
import numpy as np
import tensorflow as tf

import model, sample, encoder

def sample_model(
    model_name='124M',
    seed=None,
    nsamples=0,
    batch_size=1,
    length=None,
    temperature=1,
    top_k=0,
    top_p=1,
    models_dir='U**r\F****y\Desktop\Python\gpt-2\models',
):
    """
    Run the sample_model
    :model_name=124M : String, which model to use
    :seed=None : Integer seed for random number generators, fix seed to
     reproduce results
    :nsamples=0 : Number of samples to return, if 0, continues to
     generate samples indefinately.
    :batch_size=1 : Number of batches (only affects speed/memory).
    :length=None : Number of tokens in generated text, if None (default), is
     determined by model hyperparameters
    :temperature=1 : Float value controlling randomness in boltzmann
     distribution. Lower temperature results in less random completions. As the
     temperature approaches zero, the model will become deterministic and
     repetitive. Higher temperature results in more random completions.
    :top_k=0 : Integer value controlling diversity. 1 means only 1 word is
     considered for each step (token), resulting in deterministic completions,
     while 40 means 40 words are considered at each step. 0 (default) is a
     special setting meaning no restrictions. 40 generally is a good value.
     :models_dir : path to parent folder containing model subfolders
     (i.e. contains the <model_name> folder)
    """
    models_dir = os.path.expanduser(os.path.expandvars(models_dir))
    enc = encoder.get_encoder(model_name, models_dir)
    hparams = model.default_hparams()
    with open(os.path.join(models_dir, model_name, 'hparams.json')) as f:
        hparams.override_from_dict(json.load(f))

    if length is None:
        length = hparams.n_ctx
    elif length > hparams.n_ctx:
        raise ValueError("Can't get samples longer than window size: %s" % hparams.n_ctx)

    with tf.Session(graph=tf.Graph()) as sess:
        np.random.seed(seed)
        tf.set_random_seed(seed)

        output = sample.sample_sequence(
            hparams=hparams, length=length,
            start_token=enc.encoder['<|endoftext|>'],
            batch_size=batch_size,
            temperature=temperature, top_k=top_k, top_p=top_p
        )[:, 1:]

        saver = tf.train.Saver()
        ckpt = tf.train.latest_checkpoint(os.path.join(models_dir, model_name))
        saver.restore(sess, ckpt)

        generated = 0
        while nsamples == 0 or generated < nsamples:
            out = sess.run(output)
            for i in range(batch_size):
                generated += batch_size
                text = enc.decode(out[i])
                print("=" * 40 + " SAMPLE " + str(generated) + " " + "=" * 40)
                print(text)

if __name__ == '__main__':
    fire.Fire(sample_model)

'''

谁能告诉我我可能做错了什么 - 我确定它确实很明显，但我已经尝试了大约 4 个小时的各种东西，但没有运气

非常感谢任何建议

Answer 1

您必须在运行脚本之前下载模型：

python3 download_model.py 124M

“download_model.py”在项目的根目录中。