【发布时间】：2018-04-13 12:56:11
【问题描述】：
我误用了 softmax 的二元交叉熵，改为分类交叉熵。并在我自己的回答中对以下问题的细节进行了一些审查
我正在尝试使用开源数据：sogou_news_csv(converted to pinyin using jieba from for text classification following https://arxiv.org/abs/1502.01710"Text Understanding from scratch" by Xiang Zhang and Yann LeCun.（主要关注使用字符级CNN的想法，但是论文中提出的结构）。
我根据 alphabet 集合 使用 one-hot 编码进行预处理，并用 0 填充所有不在 alphabet 集合 中的编码。 结果，我得到了形状为 (450000, 1000, 70),(data_size, sequence_length, alphabet_size) 的训练数据。
然后我将数据输入到 http://www.wildml.com/2015/12/implementing-a-cnn-for-text-classification-in-tensorflow/ 之后的 cnn 结构中。
问题是
在训练过程中，loss和acc只发生了变化，我再次尝试对数据进行预处理，tried different learning rate settings，但没有帮助，请问出了什么问题？
下面是one-hot编码：
import numpy as np
all_letters = "abcdefghijklmnopqrstuvwxyz0123456789-,;.!?:'\"/\\|_@#$%^&*~`+-=<>()[]{}\n"
n_letters = len(all_letters)
def letterToIndex(letter):
"""
'c' -> 2
"""
return all_letters.find(letter)
def sets2tensors(clean_train, n_letters=n_letters, MAX_SEQUENCE_LENGTH=1000):
"""
From lists of cleaned passages to np.array with shape(len(train),
max_sequence_length, len(dict))
Arg:
obviously
"""
m = len(clean_train)
x_data = np.zeros((m, MAX_SEQUENCE_LENGTH, n_letters))
for ix in range(m):
for no, letter in enumerate(clean_train[ix]):
if no >= 1000:
break
letter_index = letterToIndex(letter)
if letter != -1:
x_data[ix][no][letter_index] = 1
else:
continue
return x_data
这是模型：
num_classes = 5
from keras.models import Sequential
from keras.layers import Activation, GlobalMaxPool1D, Merge, concatenate, Conv1D, Dense, Dropout
from keras.callbacks import EarlyStopping
from keras.optimizers import SGD
submodels = []
for kw in (3, 4, 5): # kernel sizes
submodel = Sequential()
submodel.add(Conv1D(32,
kw,
padding='valid',
activation='relu',
strides=1, input_shape=(1000, n_letters)))
submodel.add(GlobalMaxPool1D())
submodels.append(submodel)
big_model = Sequential()
big_model.add(Merge(submodels, mode="concat"))
big_model.add(Dense(64))
big_model.add(Dropout(0.5))
big_model.add(Activation('relu'))
big_model.add(Dense(num_classes))
big_model.add(Activation('softmax'))
print('Compiling model')
opt = SGD(lr=1e-6) # tried different learning rate from 1e-6 to 1e-1
# changed from binary crossentropy to categorical_crossentropy
big_model.compile(loss='categorical_crossentropy',
optimizer=opt,
metrics=['accuracy'])
一些结果
Train on 5000 samples, validate on 5000 samples
Epoch 1/5
5000/5000 [==============================] - 54s - loss: 0.5198 - acc: 0.7960 - val_loss: 0.5001 - val_acc: 0.8000
Epoch 2/5
5000/5000 [==============================] - 56s - loss: 0.5172 - acc: 0.7959 - val_loss: 0.5000 - val_acc: 0.8000
Epoch 3/5
5000/5000 [==============================] - 56s - loss: 0.5198 - acc: 0.7965 - val_loss: 0.5000 - val_acc: 0.8000
Epoch 4/5
5000/5000 [==============================] - 57s - loss: 0.5222 - acc: 0.7950 - val_loss: 0.4999 - val_acc: 0.8000
Epoch 5/5
5000/5000 [==============================] - 59s - loss: 0.5179 - acc: 0.7960 - val_loss: 0.4999 - val_acc: 0.8000
【问题讨论】：
标签： nlp keras conv-neural-network