我的 tensorflow 神经网络精度没有变化

Question

我想为Student Admission dataset(admit, gre, gpa, rank) 建立一个神经网络 我承认并排名如下

    one_hot_data = pd.concat([data, pd.get_dummies(data['rank'], prefix='rank')], axis=1)
    one_hot_data = pd.concat([one_hot_data, pd.get_dummies(data['admit'], prefix='admit')], axis=1)
    
    # Drop the previous rank column
    data = one_hot_data.drop('rank', axis=1)
    data = one_hot_data.drop('admit', axis=1)
    print(data.shape)

我使用train_test_split 拆分数据并使用minmax_scale 进行缩放 但是神经网络如下

n_features = X_train.shape[1]
n_labels = y_train.shape[1]

features = tf.placeholder(tf.float32, [None, n_features])
labels = tf.placeholder(tf.float32, [None, n_labels])

w = [
    tf.Variable(tf.random_normal((n_features, 16)), name='Weights_layer_0'),
    tf.Variable(tf.random_normal((16, 4)), name='Weights_layer_1'),
    tf.Variable(tf.random_normal((4, n_labels)), name='Weights_layer_2'),
]

n_layers = len(w)
b = [
    tf.Variable(tf.zeros(16), name='Bias_layer_0'),
    tf.Variable(tf.zeros(4), name='Bias_layer_1'),
    tf.Variable(tf.zeros(n_labels), name='Bias_layer_2'),
]
def neural_network(input, weights, biases):
    for i in range(n_layers-1):
        layer = tf.add(tf.matmul(input if i==0 else layer, weights[i]),biases[i])
        layer = tf.nn.relu(layer)
        # layer = tf.nn.dropout(layer, keep_prob=0.6)
    out_layer = tf.add(tf.matmul(layer, weights[-1]),biases[-1])
    return out_layer

loss_ = []
res = []
prediction = neural_network(features, w, b)
loss = tf.reduce_mean(tf.nn.sigmoid_cross_entropy_with_logits(logits=prediction, labels=labels))
optim = tf.train.AdadeltaOptimizer(0.0001).minimize(loss)
correct_prediction = tf.equal(tf.argmax(prediction, 1), tf.argmax(labels, 1))
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
with tf.device('/gpu'):
    with tf.Session() as sess:
        sess.run(tf.global_variables_initializer())
        for i in range(10):
            for m,n in zip(X_train_batches, y_train_batches):
                _, l = sess.run([optim, loss],feed_dict={features: m, labels: n})
            loss_.append(l)

            acc = sess.run([accuracy], feed_dict={features: X_train, labels: y_train})
            print(i, acc)
        test_accuracy = sess.run(accuracy,feed_dict={features: X_test, labels: y_test})
        print(test_accuracy)
        res = sess.run(neural_network(features,w,b),feed_dict={features: X})

但准确性不会改变

0 [0.4857143]
1 [0.4857143]
2 [0.4857143]
3 [0.4857143]
4 [0.4857143]
5 [0.4857143]
6 [0.4857143]
7 [0.4857143]
8 [0.4857143]
9 [0.4857143]
10 [0.4857143]
0.5333333

损失保持不变

[0.5546836, 0.5546756, 0.5546678, 0.55466014, 0.55465263, 0.5546452, 0.55463773, 0.55463034, 0.5546232, 0.5546159, 0.5546088, 0.5546016, 0.5545944, 0.5545874, 0.5545803, 0.5545734, 0.55456626, 0.5545592, 0.5545522, 0.5545452]

缺少什么？我的神经网络正确吗？ Full code

Answer 1

这里可能有很多可能的原因（我们没有你的数据），但是根据我的经验，在这种情况下，一个常见的错误是在tf.random_normal() 中使用默认参数stddev=1.0 初始化权重（见docs），就像你在这里做的那样。

stddev=1.0 是一个巨大 值，它本身就可以让你的 NN 误入歧途。将所有初始权重更改为 stddev=0.01：

w = [
    tf.Variable(tf.random_normal((n_features, 16), stddev=0.01), name='Weights_layer_0'),
    tf.Variable(tf.random_normal((16, 4), stddev=0.01), name='Weights_layer_1'),
    tf.Variable(tf.random_normal((4, n_labels), stddev=0.01), name='Weights_layer_2'),
    ]

除此之外，正如 cmets 中已经建议的那样，0.0001 的学习率在这里似乎太小了（考虑到损失下降的速度有多慢）；尝试更高的值 (0.01 - 0.001)。