在 TensorFlow 和 Keras 中运行相同的神经网络时结果不一致

Question

我为 MNIST 数据集创建了两个相同的神经网络，一个使用 TensorFlow，一个使用 Keras。在 10 个 epoch 时，Keras 的性能达到了 96% 以上，而 TensorFlow 达到了大约 70%。

我也在其他数据集上测试了下面的代码，在直接参数比较中，TensorFlow 在所有这些数据集上的性能要低得多。

Keras 代码：

import warnings
warnings.filterwarnings('ignore')
import keras
from keras.datasets import mnist

# Loading MNIST
(x_train, y_train), (x_test, y_test) = mnist.load_data() 

# Converting the y-value column to an array of classes (one hot enconding)
from keras.utils import np_utils
y_train = np_utils.to_categorical(y_train) 
y_test = np_utils.to_categorical(y_test)

# Changing the shape of input images and normalizing
x_train = x_train.reshape((60000, 784))
x_train = x_train.astype('float32') / 255
x_test = x_test.reshape((10000, 784))
x_test = x_test.astype('float32') / 255

# Making the neural network
from keras.models import Sequential
from keras.layers import Dense, Activation

model = Sequential()
model.add(Dense(30, input_dim=784, kernel_initializer='normal', activation='relu'))
model.add(Dense(30, kernel_initializer='normal', activation='relu')) 
model.add(Dense(10, kernel_initializer='normal', activation='softmax')) 

from keras.optimizers import Adam
optimizer = Adam()
model.compile(loss='categorical_crossentropy', optimizer=optimizer, metrics=['acc'])

# Training and showing the results
model.fit(x_train, y_train, epochs=10, batch_size=200, validation_data=(x_test, y_test), verbose=1)

TensorFlow 代码：

import warnings
warnings.filterwarnings('ignore')
import tensorflow as tf

#Loading MNIST
from tensorflow.examples.tutorials.mnist import input_data
mnist = input_data.read_data_sets("MNIST_data/", one_hot=True)

# Epochs parameters
epochs = 10
batch_size = 200

# Neural network parameters
n_input = 784 
n_hidden_1 = 30 
n_hidden_2 = 30 
n_classes = 10 

# Placeholders x, y
x = tf.placeholder(tf.float32, [None, n_input])
y = tf.placeholder(tf.float32, [None, n_classes])

# Creating the first layer 
w1 = tf.Variable(tf.random_normal([n_input, n_hidden_1]))
b1 = tf.Variable(tf.random_normal([n_hidden_1]))
layer_1 = tf.nn.relu(tf.add(tf.matmul(x,w1),b1))

# Creating the second layer 
w2 = tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2]))
b2 = tf.Variable(tf.random_normal([n_hidden_2]))
layer_2 = tf.nn.relu(tf.add(tf.matmul(layer_1,w2),b2)) 

# Creating the output layer 
w_out = tf.Variable(tf.random_normal([n_hidden_2, n_classes]))
bias_out = tf.Variable(tf.random_normal([n_classes]))
output = tf.add(tf.matmul(layer_2, w_out), bias_out)

# Loss function
cost = tf.reduce_mean(tf.nn.softmax_cross_entropy_with_logits(logits = output, labels = y))
# Optimizer
optimizer = tf.train.AdamOptimizer().minimize(cost)

# Making predictions
predictions = tf.equal(tf.argmax(output, 1), tf.argmax(y, 1))

# Accuracy
accuracy = tf.reduce_mean(tf.cast(predictions, tf.float32))

# Initializing the variables
init = tf.global_variables_initializer()

# Opening the session
with tf.Session() as sess:
    sess.run(init)  
    # Training cycle
    for epoch in range(epochs):
        avg_cost = 0.0
        total_batches = int(mnist.train.num_examples / batch_size)

        # Loop through all batch iterations
        for i in range(total_batches):
            batch_x, batch_y = mnist.train.next_batch(batch_size)

            # Fit training
            sess.run(optimizer, feed_dict={x: batch_x, y: batch_y})

            # Computing the average cost of a complete epoch
            avg_cost += sess.run(cost, feed_dict={x: batch_x, y: batch_y}) / total_batches

        # Running accuracy (with test data) on each epoch
        accuracy_test = sess.run(accuracy, feed_dict={x: mnist.test.images, y: mnist.test.labels})

        # Showing results after each epoch
        print ("Epoch: ", "{},".format((epoch + 1)), "Average cost = ", "{:.3f}".format(avg_cost))
        print ("Accuracy Test = ", "{:.3f}".format(accuracy_test))  
    print ("Training completed!")
    print ("Model Accuracy:", accuracy.eval({x: mnist.test.images, y: mnist.test.labels}))

谁能帮我了解导致这种分歧的原因？

Answer 1

归一化极大地提高了模型的准确性。

在 keras 代码中，您已经对数据进行了规范化，但是我找不到在 tensorflow 代码中完成的任何规范化，或者 tensorflow 中的 mnist 数据可能已经在源代码上进行了规范化？

请验证tensorflow代码中的mnist数据也被规范化了