多层神经网络 - 损失函数为负且准确度（低）保持不变

Question

我构建了一个有两个隐藏层的神经网络。我对两层使用 ReLu 激活，对于最后一层（out_layer）我使用线性激活函数。输入值 x 有 125 列（特征）。型号为：

def multilayer_perceptron():
    tf.reset_default_graph()
    x = tf.placeholder(tf.float32, shape=[None,train_x.shape[1]])
    y = tf.placeholder(tf.float32, shape=[None, 1])
    weights = {
    'h1': tf.Variable(tf.random_normal([train_x.shape[1], n_hidden_1])),
    'h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
    'out': tf.Variable(tf.random_normal([n_hidden_2, 1]))
    }
    biases = {
    'b1': tf.Variable(tf.random_normal([n_hidden_1])),
    'b2': tf.Variable(tf.random_normal([n_hidden_2])),
    'out': tf.Variable(tf.random_normal([1]))
    }
    # Hidden layer with ReLU
    layer_1 = tf.add(tf.matmul(x, weights['h1']), biases['b1'])
    layer_1 = tf.nn.relu(layer_1)
    # Hidden layer with ReLU 
    layer_2 = tf.add(tf.matmul(layer_1, weights['h2']), biases['b2'])
    layer_2 = tf.nn.relu(layer_2)
    # Output layer with linear activation
    out_layer = tf.matmul(layer_2, weights['out']) + biases['out']
    learning_rate = tf.placeholder(tf.float32)
    is_training=tf.Variable(True,dtype=tf.bool) 
    cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=out_layer,logits=y )
    cost = tf.reduce_mean(cross_entropy)  
    with tf.control_dependencies(tf.get_collection(tf.GraphKeys.UPDATE_OPS)):
         optimizer = tf.train.AdamOptimizer(learning_rate=learning_rate).minimize(cost)
    predicted = tf.nn.sigmoid(out_layer) 
    correct_pred = tf.equal(tf.round(predicted), y)
    accuracy = tf.reduce_mean(tf.cast(correct_pred, tf.float32))
    # Export the nodes 
    export_nodes = ['x', 'y', 'learning_rate','is_training', 'out_layer',
                    'cost', 'optimizer', 'predicted',  'accuracy'] 
    Graph = namedtuple('Graph', export_nodes)
    local_dict = locals()
    graph = Graph(*[local_dict[each] for each in export_nodes])
    return graph

pred1 = multilayer_perceptron()

定义训练集和有效集的损失和准确度的会话输出如下：

Epoch: 1/25 Train Loss: -219676.2812 Train Acc: 0.1047
....
....
Epoch: 7/25 Train Loss: -46358476.0000 Train Acc: 0.1047
Epoch: 7/25 Validation Loss: -46845576.0000 Validation Acc: 0.1038
Epoch: 7/25 Train Loss: -53906272.0000 Train Acc: 0.1047
Epoch: 7/25 Validation Loss: -54461228.0000 Validation Acc: 0.1038
Epoch: 7/25 Train Loss: -62156116.0000 Train Acc: 0.1047
Epoch: 7/25 Validation Loss: -62784408.0000 Validation Acc: 0.1038
...
...
Epoch: 13/25 Train Loss: -304887584.0000 Train Acc: 0.1047
Epoch: 13/25 Validation Loss: -307502528.0000 Validation Acc: 0.1038
Epoch: 13/25 Train Loss: -327771616.0000 Train Acc: 0.1047
Epoch: 13/25 Validation Loss: -330566752.0000 Validation Acc: 0.1038
Epoch: 13/25 Train Loss: -351641632.0000 Train Acc: 0.1047
Epoch: 13/25 Validation Loss: -354622816.0000 Validation Acc: 0.1038
...
...
Epoch: 25/25 Train Loss: -1748839296.0000 Train Acc: 0.1047
Epoch: 25/25 Validation Loss: -1762260480.0000 Validation Acc: 0.1038
Epoch: 25/25 Train Loss: -1811036800.0000 Train Acc: 0.1047
Epoch: 25/25 Validation Loss: -1824913024.0000 Validation Acc: 0.1038
Epoch: 25/25 Train Loss: -1874472576.0000 Train Acc: 0.1047
Epoch: 25/25 Validation Loss: -1888812416.0000 Validation Acc: 0.1038
'./insurance2.ckpt'

我认为这个模型没有学习。

Answer 1

你已经反转了交叉熵的标签和 logits 参数。

cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=out_layer,logits=y)

应该是

cross_entropy = tf.nn.sigmoid_cross_entropy_with_logits(labels=y, logits= out_layer)

你的代码应该可以正常工作