【发布时间】：2021-08-16 20:44:40
【问题描述】：
我在 python 中实现了一个 chatbot，它使用“intents”数据集进行训练，该数据集是一个 json 文件，格式如下：
{"intents": [
{"tag": "greeting",
"patterns": ["Hi there", "How are you", "Is anyone there?","Hey","Hola", "Hello", "Good day"],
"responses": ["Hello, thanks for asking", "Good to see you again", "Hi there, how can I help?"],
},
{"tag": "goodbye",
"patterns": ["Bye", "See you later", "Goodbye", "Nice chatting to you, bye", "Till next time"],
"responses": ["See you!", "Have a nice day", "Bye! Come back again soon."],
},
{"tag": "thanks",
"patterns": ["Thanks", "Thank you", "That's helpful", "Awesome, thanks", "Thanks for helping me"],
"responses": ["Happy to help!", "Any time!", "My pleasure"],
},
{"tag": "noanswer",
"patterns": [],
"responses": ["Sorry, can't understand you", "Please give me more info", "Not sure I understand"],
.
.
.
其中标签是用户问题的类别（patterns）以及相关的可能的responses。 在训练阶段之前，数据集已被转换，使用 tokenization 提取模式的每个单词，然后应用 lemmatization。因此，训练集由带有相关标签（标签）的模式组成，其中模式表示为 Bag of Words，标签使用 one-hot 编码进行编码。 然后模型被定义如下：
model = Sequential()
model.add(Dense(128, input_shape=(x_train.shape[1],), activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(64, activation='relu'))
model.add(Dropout(0.5))
model.add(Dense(len(classes), activation="softmax"))
# set the optimizer
sgd = SGD(lr=0.01, decay=1e-6, momentum=0.9, nesterov=True)
# compile the model
model.compile(loss='categorical_crossentropy', optimizer=sgd, metrics=['accuracy'])
训练了 500 个 epoch，批量大小为 16。
分类效果很好，模型能够在给定正确“标签”的情况下正确分类看不见的问题。如果预测概率高于 0.75，则模型返回正确的标签，否则返回标签“noanswer”。
问题是当我向聊天机器人询问一个故意错误的问题时，写一个像“fejfeajlflnk”或类似的随机字符串来测试在哪种情况下返回的标签是“noanswer”（低预测概率，低于 0.75 ）分类总是以高概率（从 0.8 到 0.99）预测与标签“问候”相关联的类，我无法理解这个事实。谁能帮我理解为什么分类器会这样？
【问题讨论】：
标签： python machine-learning keras nlp chatbot