RNN 输出和基于规则的输出有什么不同？

Question

我是机器学习的新手，我有一个问题，我是 following this tutorial，我阅读了有关 LSTM 和 RNN 的信息。我使用教程提供的代码并运行它，它完成了训练，现在我提供了一些字符串进行测试：

Training data is this :

输出是：

Iter= 20000, Average Loss= 0.531466, Average Accuracy= 84.60%
['the', 'sly', 'and'] - [treacherous] vs [treacherous]
Optimization Finished!
Elapsed time:  12.159853319327036 min
Run on command line.
    tensorboard --logdir=/tmp/tensorflow/rnn_words
Point your web browser to: http://localhost:6006/
3 words: ,hello wow and
Word not in dictionary
3 words: mouse,mouse,mouse
3 words: mouse
3 words: mouse mouse mouse
mouse mouse mouse very well , but who is to bell the cat approaches the until will at one another and take mouse a receive some signal of her approach , we he easily escape
3 words: 3 words: had a general
had a general to proposal to make round the neck will all agree , said he easily at and enemy approaches to consider what common the case . you will all agree , said he
3 words: mouse mouse mouse
mouse mouse mouse very well , but who is to bell the cat approaches the until will at one another and take mouse a receive some signal of her approach , we he easily escape
3 words: what was cat
what was cat up and said he is all very well , but who is to bell the cat approaches the until will at one another and take mouse a receive some signal of her
3 words: mouse fear cat
Word not in dictionary
3 words: mouse tell cat
Word not in dictionary
mo3 words: mouse said cat
Word not in dictionary
3 words: mouse fear fear
Word not in dictionary
3 words: mouse ring bell
Word not in dictionary
m3 words: mouse ring ring
Word not in dictionary
3 words: mouse bell bell
mouse bell bell and general to make round the neck will all agree , said he easily at and enemy approaches to consider what common the case . you will all agree , said he
3 words: mouse and bell
mouse and bell this means we should always , but looked is young always , but looked is young always , but looked is young always , but looked is young always , but looked
3 words: mouse was bell
mouse was bell and said he is all very well , but who is to bell the cat approaches the until will at one another and take mouse a receive some signal of her approach
3 words: 

现在我没有得到什么，当我给出三个单词时，它会给出类似的结果，我们可以通过正则表达式或使用 if-else 的基于规则的代码轻松实现，例如文件中的输入单词然后获取前一个或下一个句子的句子, 这个输出有什么特别之处，它有什么不同？请解释一下

就像有时它说的那样，单词不在 dict 中，所以如果我必须只给出训练文件中的那些单词，那么就像它在训练数据中匹配 inout 单词并从文件中获取一些结果一样，那么我们可以做同样的事情if else 或者在没有任何模块的纯编程中，那么它有什么不同？

Answer 1

您的训练数据集只有大约 180 个单词，并且达到了 84.6%（训练）的准确率，因此它有点过度拟合。本质上，该模型只是根据训练数据预测下一个最有可能出现的词。

通常在更大的数据集上训练语言模型，例如 PTB 或 1B 单词基准。 PTB 是一个小数据集，有 100,000 个单词，而 1B word benchmark 有 10 亿个单词。

RNN 模型的词汇量有限，无法对单词或字符进行编码。词汇量取决于模型。大多数在 PTB 上训练的词模型的词汇量为 10,000，这对于大多数常见词来说已经足够了。