为什么神经网络不学习？

Question

我正在用一个简单的数据集训练一个神经网络。我尝试了参数、优化器、学习率的不同组合……但即使经过 20 个 epoch，网络仍然没有学到任何东西。

不知道下面的代码哪里出了问题？

from tensorflow.keras.models import  Sequential, load_model
from tensorflow.keras.layers import Input, Dense, Flatten
from tensorflow import keras
from livelossplot import PlotLossesKeras
from keras.models import Model
from sklearn.datasets import make_classification
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
import pandas as pd


seed = 42

X, y = make_classification(n_samples=100000, n_features=2, n_redundant=0, 
                           n_informative=2, random_state=seed)

print(f"Number of features: {X.shape[1]}")
print(f"Number of samples: {X.shape[0]}")


df = pd.DataFrame(np.concatenate((X,y.reshape(-1,1)), axis=1))
df.set_axis([*df.columns[:-1], 'Class'], axis=1, inplace=True)

df['Class'] = df['Class'].astype('int')
X = df.drop('Class', axis=1)
y = df['Class']

X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2, random_state=42)

print(f"Train set: {X_train.shape}")
print(f"Validation set: {X_val.shape}")


scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train.astype(np.float64))
X_val_scaled = scaler.transform(X_val.astype(np.float64))

inputs = Input(shape=X_train_scaled.shape[1:])
h0 = Dense(5, activation='relu')(inputs)
h1 = Dense(5, activation='relu')(h0)
preds = Dense(1, activation = 'sigmoid')(h1)

model = Model(inputs=inputs, outputs=preds)
opt = keras.optimizers.Adam(lr=0.0001)
model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])
history = model.fit(X_train_scaled, y_train, batch_size=128, epochs=20, verbose=0,
                    validation_data=(X_val_scaled, y_val),
                    callbacks=[PlotLossesKeras()]) 

score_train = model.evaluate(X_train_scaled, y_train, verbose=0)
score_test = model.evaluate(X_val_scaled, y_val, verbose=0) 
print('Train score:', score_train[0])
print('Train accuracy:', score_train[1])
print('Test score:', score_test[0])
print('Test accuracy:', score_test[1])

代码产生以下类型的输出

Answer 1

你使用了错误的损失函数，改变这一行

model.compile(optimizer=opt, loss='categorical_crossentropy', metrics=['accuracy'])

例如，

model.compile(optimizer=opt, loss='mse', metrics=['accuracy'])

分类交叉熵需要一个单热编码的y，这意味着每个类都必须有一个0 或1。 MSE 只是均方误差，所以它会起作用。但是您也可以尝试其他一些损失。

你的y:

[1,0,1]

单热编码y:

[[0,1], [1,0], [0,1]]