【发布时间】：2018-07-04 15:26:05
【问题描述】：
我正在尝试在一些数据上学习一个简单的线性 softmax 模型。 scikit-learn 中的 LogisticRegression 似乎工作正常，现在我正在尝试将代码移植到 TensorFlow，但我没有得到相同的性能，但更差了一些。我知道结果不会完全相等（scikit learn 有正则化参数等），但相差太远了。
total = pd.read_feather('testfile.feather')
labels = total['labels']
features = total[['f1', 'f2']]
print(labels.shape)
print(features.shape)
classifier = linear_model.LogisticRegression(C=1e5, solver='newton-cg', multi_class='multinomial')
classifier.fit(features, labels)
pred_labels = classifier.predict(features)
print("SCI-KITLEARN RESULTS: ")
print('\tAccuracy:', classifier.score(features, labels))
print('\tPrecision:', precision_score(labels, pred_labels, average='macro'))
print('\tRecall:', recall_score(labels, pred_labels, average='macro'))
print('\tF1:', f1_score(labels, pred_labels, average='macro'))
# now try softmax regression with tensorflow
print("\n\nTENSORFLOW RESULTS: ")
## By default, the OneHotEncoder class will return a more efficient sparse encoding.
## This may not be suitable for some applications, such as use with the Keras deep learning library.
## In this case, we disabled the sparse return type by setting the sparse=False argument.
enc = OneHotEncoder(sparse=False)
enc.fit(labels.values.reshape(len(labels), 1)) # Reshape is required as Encoder expect 2D data as input
labels_one_hot = enc.transform(labels.values.reshape(len(labels), 1))
# tf Graph Input
x = tf.placeholder(tf.float32, [None, 2]) # 2 input features
y = tf.placeholder(tf.float32, [None, 5]) # 5 output classes
# Set model weights
W = tf.Variable(tf.zeros([2, 5]))
b = tf.Variable(tf.zeros([5]))
# Construct model
pred = tf.nn.softmax(tf.matmul(x, W) + b) # Softmax
clas = tf.argmax(pred, axis=1)
# Minimize error using cross entropy
cost = tf.reduce_mean(-tf.reduce_sum(y*tf.log(pred), reduction_indices=1))
# Gradient Descent
optimizer = tf.train.GradientDescentOptimizer(0.01).minimize(cost)
# Initialize the variables (i.e. assign their default value)
init = tf.global_variables_initializer()
# Start training
with tf.Session() as sess:
# Run the initializer
sess.run(init)
# Training cycle
for epoch in range(1000):
# Run optimization op (backprop) and cost op (to get loss value)
_, c = sess.run([optimizer, cost], feed_dict={x: features, y: labels_one_hot})
# Test model
correct_prediction = tf.equal(tf.argmax(pred, 1), tf.argmax(y, 1))
class_out = clas.eval({x: features})
# Calculate accuracy
accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
print("\tAccuracy:", accuracy.eval({x: features, y: labels_one_hot}))
print('\tPrecision:', precision_score(labels, class_out, average='macro'))
print('\tRecall:', recall_score(labels, class_out, average='macro'))
print('\tF1:', f1_score(labels, class_out, average='macro'))
这段代码的输出是
(1681,)
(1681, 2)
SCI-KITLEARN RESULTS:
Accuracy: 0.822129684711
Precision: 0.837883361162
Recall: 0.784522522208
F1: 0.806251963817
TENSORFLOW RESULTS:
Accuracy: 0.694825
Precision: 0.735883666192
Recall: 0.649145125846
F1: 0.678045562185
我检查了 one-hot-encoding 的结果和数据，但我不知道为什么 TF 中的结果要差得多。
任何建议都会非常感激..
【问题讨论】：
-
至少有一个很大的区别：
newton_cg是在做线搜索，你的 tensorflow 代码没有 -
Maxim 的观点很有趣，我没想到。我对你的损失函数有点失望，我知道它不是交叉熵的形式。
−(y log(pred)+(1−y)log(1−pred))我想你只定义了一半的交叉熵损失函数。或者也许我只是不熟悉那种形式？只是为了确保我会使用tf.nn.softmax_cross_entropy_with_logits来验证事情。 -
@David Parks 嗯，我使用了与 TensorFlow 的 softmax 教程中使用的相同的损失函数 (tensorflow.org/get_started/mnist/beginners)。我确实注意到降低学习率和增加 epoch 会给我带来相同的结果！
-
@David Parks tf.softmax_cross_entropy_with_logits 给出的结果与上面描述的结果相同。不过，根据文档，后者更健壮
标签： tensorflow machine-learning scikit-learn logistic-regression softmax