三层前馈神经网络不能准确预测回归值

Question

我对 Tensorflow 还是很陌生。目前，我正在做一个 3 层网络，隐藏层中有 10 个神经元，ReLU，小批量梯度下降大小为 8，L2 正则化权重衰减参数 (beta) 为 0.001。我使用的 Tensorflow 版本是 1.14，我使用的是 Python 3.6。

让我难以置信的问题是我的预测值和测试错误绝对超出图表。

例如，我绘制了样本大小为 50 and this is what came out. 的测试错误和预测值 vs 目标值 正如你所看到的，这两个情节都离我们很远，我对为什么没有丝毫线索。

Here's how the dataset roughly looks like.第一列被丢弃，因为它只是一个计数器值，最后一列是目标。

我的代码：

NUM_FEATURES = 7
num_neuron = 10
batch_size = 8
beta = 0.001
learning_rate = 0.001

epochs = 4000
seed = 10
np.random.seed(seed)

# read and divide data into test and train sets
total_dataset= np.genfromtxt('dataset_excel.csv', delimiter=',')
X_data, Y_data = total_dataset[1:, 1:8], total_dataset[1:, -1]
Y_data = Y_data.reshape(Y_data.shape[0], 1)

# shuffle input, ensure both are shuffled with the same order
shufflestate = np.random.get_state()
np.random.shuffle(X_data)
np.random.set_state(shufflestate)
np.random.shuffle(Y_data)

# 70% used for training, 30% used for testing
trainX = X_data[:280]
trainY = Y_data[:280]
testX = X_data[280:]
testY = Y_data[280:]

trainX = (trainX - np.mean(trainX, axis=0)) / np.std(trainX, axis=0)

# Create the model
x = tf.placeholder(tf.float32, [None, NUM_FEATURES])
y_ = tf.placeholder(tf.float32, [None, 1])

# get 50 samples for plotting of predicted vs target values
limited50testX = testX[:50]
limited50testY = testY[:50]

# Hidden
with tf.name_scope('hidden'):
    weight1 = tf.Variable(tf.truncated_normal([NUM_FEATURES, num_neuron],stddev=1.0,name='weight1'))
    bias1 = tf.Variable(tf.zeros([num_neuron]),name='bias1')
    hidden = tf.nn.relu(tf.matmul(x, weight1) + bias1)

# output
with tf.name_scope('linear'):
    weight2 = tf.Variable(tf.truncated_normal([num_neuron, 1],stddev=1.0 / np.sqrt(float(num_neuron))),name='weight2')
    bias2 = tf.Variable(tf.zeros([1]),name='bias2')
    logits = tf.matmul(hidden, weight2) + bias2

ridgeLoss = tf.square(y_ - logits)
regularisation = tf.nn.l2_loss(weight1) + tf.nn.l2_loss(weight2)
loss = tf.reduce_mean(ridgeLoss + beta * regularisation)

optimizer = tf.train.GradientDescentOptimizer(learning_rate)
train_op = optimizer.minimize(loss)
error = tf.reduce_mean(tf.square(y_ - logits))

N = len(trainX)
idx = np.arange(N)

predicted=[]

with tf.Session() as sess:
    sess.run(tf.global_variables_initializer())
    train_err = []
    test_err_ = []
    
    for i in range(epochs):
        for batchStart, batchEnd in zip(range(0, trainX.shape[0], batch_size),range(batch_size, trainX.shape[0], batch_size)):
            train_op.run(feed_dict={x: trainX[batchStart:batchEnd], y_: trainY[batchStart:batchEnd]})
        err = error.eval(feed_dict={x: trainX, y_: trainY})
        train_err.append(err)
        
            
        if i % 100 == 0:
            print('iter %d: train error %g' % (i, train_err[i]))
        test_err = error.eval(feed_dict={x: testX, y_: testY})
        test_err_.append(test_err)

    predicted = sess.run(logits, feed_dict={x:limited50testX})
    print("predicted values: ", predicted)
    print("size of predicted values is", len(predicted))
    
    print("targets: ", limited50testY)
    print("size of target values is", len(limited50testY))

#plot predictions vs targets
numberList=np.arange(0, 50, 1).tolist()
predplot = plt.figure(1)
plt.plot(numberList, predicted, label='Predictions')
plt.plot(numberList, limited50testY, label='Targets')
plt.xlabel('50 samples')
plt.ylabel('Value')
plt.legend(loc='lower right')
predplot.show()

# plot training error
trainplot = plt.figure(2)
plt.plot(range(epochs), train_err)
plt.xlabel(str(epochs) + ' iterations')
plt.ylabel('Train Error')
trainplot.show()

#plot testing error
testplot = plt.figure(3)
plt.plot(range(epochs), test_err_)
plt.xlabel(str(epochs) + ' iterations')
plt.ylabel('Test Error')
testplot.show()

Answer 1

不确定是不是这样，但trainX 是标准化的，而testX 不是。在预测之前，您可能希望在 testX 上使用相同的归一化。