【问题标题】:How do I output this Python, jyputer, deepplavlov code correctly on a notebook cell?如何在笔记本单元格上正确输出此 Python、jupyter、deeppavlov 代码?
【发布时间】:2020-04-20 05:27:45
【问题描述】:

我有一个使用 Tensorflow 和 Jupyter 的功能设置。我已将 Tensorflow==1.14 配置为在 gpu 上运行。

现在问题: 我正在使用一个名为 DeepPavlov 的开源对话式 AI 框架。它全部启动并运行(在配置方面),但我没有太多从笔记本调用 python 的经验(或根本没有)。我可以在控制台上运行此代码,但这不是我的目标。我的问题:

我有一个普通的python代码:

from deeppavlov import build_model, configs

model = build_model(configs.squad.squad, download=True)
model(['DeepPavlov is library for NLP and dialog systems.'], ['What is DeepPavlov?'])

这是我的输出:

2020-04-20 07:08:23.478 INFO in 'deeppavlov.download'['download'] at line 117: Skipped http://files.deeppavlov.ai/deeppavlov_data/squad_model_1.4_cpu_compatible.tar.gz download because of matching hashes
2020-04-20 07:08:37.884 INFO in 'deeppavlov.download'['download'] at line 117: Skipped http://files.deeppavlov.ai/embeddings/wiki-news-300d-1M.vec download because of matching hashes
2020-04-20 07:08:38.343 INFO in 'deeppavlov.download'['download'] at line 117: Skipped http://files.deeppavlov.ai/embeddings/wiki-news-300d-1M-char.vec download because of matching hashes
2020-04-20 07:08:38.364 INFO in 'deeppavlov.models.preprocessors.squad_preprocessor'['squad_preprocessor'] at line 310: SquadVocabEmbedder: loading saved tokens vocab from C:\Users\Administrator\.deeppavlov\models\squad_model\emb\vocab_embedder.pckl
2020-04-20 07:08:39.158 INFO in 'deeppavlov.models.preprocessors.squad_preprocessor'['squad_preprocessor'] at line 310: SquadVocabEmbedder: loading saved chars vocab from C:\Users\Administrator\.deeppavlov\models\squad_model\emb\char_vocab_embedder.pckl
2020-04-20 07:08:40.599 INFO in 'deeppavlov.core.layers.tf_layers'['tf_layers'] at line 615: 
Warning! tf.contrib.cudnn_rnn.CudnnCompatibleGRUCell is used. It is okay for inference mode, but if you train your model with this cell it could NOT be used with tf.contrib.cudnn_rnn.CudnnGRUCell later. 
2020-04-20 07:08:41.185 INFO in 'deeppavlov.core.layers.tf_layers'['tf_layers'] at line 615: 
Warning! tf.contrib.cudnn_rnn.CudnnCompatibleGRUCell is used. It is okay for inference mode, but if you train your model with this cell it could NOT be used with tf.contrib.cudnn_rnn.CudnnGRUCell later. 
2020-04-20 07:08:41.520 INFO in 'deeppavlov.core.layers.tf_layers'['tf_layers'] at line 615: 
Warning! tf.contrib.cudnn_rnn.CudnnCompatibleGRUCell is used. It is okay for inference mode, but if you train your model with this cell it could NOT be used with tf.contrib.cudnn_rnn.CudnnGRUCell later. 
2020-04-20 07:08:41.748 INFO in 'deeppavlov.core.layers.tf_layers'['tf_layers'] at line 615: 
Warning! tf.contrib.cudnn_rnn.CudnnCompatibleGRUCell is used. It is okay for inference mode, but if you train your model with this cell it could NOT be used with tf.contrib.cudnn_rnn.CudnnGRUCell later. 
2020-04-20 07:09:23.205 INFO in 'deeppavlov.core.models.tf_model'['tf_model'] at line 51: [loading model from C:\Users\Administrator\.deeppavlov\models\squad_model\model]
INFO:tensorflow:Restoring parameters from C:\Users\Administrator\.deeppavlov\models\squad_model\model
[['library for NLP and dialog systems'], [14], [8040850.5]]

它正常运行,但没有给我任何交互选项,如提示或它只是停止的东西。

我的目标是获得可以输入和输出(文本等)的提示。我知道我可能在使用 python 笔记本/单元时犯了一些非常基本的错误,如果您需要更多信息,请询问。谢谢。

【问题讨论】:

    标签: python python-3.x tensorflow jupyter-notebook cell


    【解决方案1】:

    DeepPavlov 附带了一组由 TensorFlow 和 Keras 提供支持的预定义组件,用于解决与 NLP 相关的问题。

    您使用的是用于问答的 BERT。 上下文问答是在给定上下文(例如,维基百科的一段)中找到问题答案的任务,其中每个问题的答案都是上下文的一部分。

    模型在调用model(contexts_list, questions_list)时返回以下内容

    answers_list, answers_starts_list, logits_list
    

    示例 1:

    from deeppavlov import build_model, configs
    model = build_model(configs.squad.squad_bert)
    model(['DeepPavlov is library for NLP and dialog systems.'], ['What is DeepPavlov?'])
    

    输出-

    INFO:tensorflow:Restoring parameters from C:\Users\RF00538236\.deeppavlov\models\squad_bert\model
    [['library for NLP and dialog systems'], [14], [158.80197143554688]]
    

    答案返回答案列表、答案开始列表(答案出现在第 14 个字符处)和 logits 列表。

    示例 2: 已从 here 复制了有关圣雄甘地的上下文。

    model(['Born and raised in a Hindu family in coastal Gujarat, western India, Gandhi was trained in law at the Inner Temple, London, and called to the bar at age 22 in June 1891. After two uncertain years in India, where he was unable to start a successful law practice, he moved to South Africa in 1893 to represent an Indian merchant in a lawsuit. He went on to stay for 21 years. It was in South Africa that Gandhi raised a family, and first employed nonviolent resistance in a campaign for civil rights.'],['What was Gandhi trained at?'])
    

    输出 -

    [['law'], [91], [106616.5390625]]
    

    或者,您可以构建 deeppavlov 与模型交互。 您可以通过在命令行中使用交互参数和模型配置文件的名称运行模型来与模型交互(-d 表示下载所有必需的文件)。否则,您可以使用 Python Jupyter 代码中的 build_model,如下例所示。

    安装依赖 -

    !python -m deeppavlov install tfidf_logreg_en_faq
    

    示例 1 -

    from deeppavlov import configs
    from deeppavlov.core.common.file import read_json
    from deeppavlov.core.commands.infer import build_model
    
    faq = build_model(configs.faq.tfidf_logreg_en_faq, download = True)
    a = faq(["I need help"])
    a
    

    输出 -

    [['If you have any further inquiries, you can address them to the International Students Office, which is located in the Auditorium Building, Room 315. The phone number is (7-495) 408-7043.'],
     [[0.0005971888349561158,
       0.0004990413077070781,
       0.0003260898111600398,
       0.0004955716039888539,
       0.9920733828654503,
       0.0004564850775432216,
       0.0012178910790545932,
       0.0006631341572001673,
       0.0006362137679856412,
       0.0010445215260383672,
       0.0011939766282353169,
       0.0004438081627736979,
       0.00035269517790671484]]]
    

    示例 2 -

    a = faq(["i need help on medical offices"])
    a
    

    输出 -

    [['All Russian universities have medical offices for first aid and general medical care.'],
     [[0.00030839735143966185,
       0.0005853193249863691,
       0.0004579660993813656,
       0.0004773336684218436,
       0.4791163218068051,
       0.00028413386610718364,
       0.0009917714916957442,
       0.00047599362403134946,
       0.000766086074333974,
       0.0007509714241461073,
       0.5120912464072629,
       0.0032745806081316926,
       0.0004198782532568704]]]
    

    希望这能回答您的问题。快乐学习。

    【讨论】:

    • 已解决,感谢您准确解释我需要知道的内容。
    • @PedrodePaula - 如果它回答了您的问题,请您投票赞成。谢谢。
    • 我没有足够的能力去做那件事
    猜你喜欢
    • 1970-01-01
    • 2020-02-06
    • 1970-01-01
    • 2019-04-28
    • 2018-01-22
    • 2021-12-31
    • 2018-03-29
    • 2021-10-12
    • 1970-01-01
    相关资源
    最近更新 更多