Keras 模型有时无法训练

Question

这是我的模型。

def get_model2():
  lr_schedule = tf.keras.optimizers.schedules.ExponentialDecay(
      0.001,
      decay_steps=100000,
      decay_rate=0.96,
      staircase=True)
 
  
 
  model = Sequential()
  model.add(Dense(1024,activation='relu',input_shape=[44]))
  model.add(Dropout(0.2))
  model.add(Dense(2048,activation='relu'))
  model.add(Dropout(0.2))
  model.add(Dense(4098,activation='relu'))
  model.add(Dense(2048,activation='relu'))
  model.add(Dense(1024,activation='relu'))
  model.add(Dense(512,activation='relu'))
  model.add(Dense(1,activation='sigmoid'))
  
  model.compile(loss=my_binary_crossentropy,optimizer=tf.keras.optimizers.Adam(learning_rate=lr_schedule),metrics=['accuracy'])
  return model

这是我的训练：

from sklearn.model_selection import RepeatedStratifiedKFold
 

model = get_model2()
model.save_weights('model.h5')
o = 0
hlavny_list = []
skf = RepeatedStratifiedKFold(n_splits=10,n_repeats=10,random_state=2027)
for train_index,test_index in skf.split(X, Y):
  o = o + 1
  X_test, X_train = X[train_index], X[test_index]
  y_test, y_train = Y[train_index], Y[test_index]
  model.load_weights('model.h5')
  model.fit(X_train,y_train,epochs=10000,batch_size=256,validation_data=(X_test,y_test),callbacks=[early_stop])
  vys = model.predict_classes(X_test)
  a,b,c,d = potrebne_miery(y_true=y_test,y_pred=vys)
  hl_list = [a,b,c,d]
  hlavny_list.append(hl_list)
  if (o % 4 == 0):
    np.savetxt('/content/drive/My Drive/siete/model_t_t_1_9_moja_loss_v2.csv',np.array(hlavny_list),delimiter=',')


np.savetxt('/content/drive/My Drive/siete/model_t_t_1_9_moja_loss_v2.csv',np.array(hlavny_list),delimiter=',')

这里没什么特别的，除了我自己的损失函数，它看起来像：

import tensorflow as tf
from tensorflow.python.ops import clip_ops

def my_binary_crossentropy(target, output, from_logits=False):

  target = ops.convert_to_tensor_v2(target)
  output = ops.convert_to_tensor_v2(output)

  target = tf.cast(target, tf.float32)


  epsilon_ = 0.01
  output = clip_ops.clip_by_value(output, epsilon_, 1. - epsilon_)

  # Compute cross entropy from probabilities.
  bce = 8 * target * math_ops.log(output + epsilon_)
  bce +=  (1 - target) * math_ops.log(1 - output + epsilon_)
  return -bce

在我使用二元交叉熵之前一切正常，没问题。但是当我改变损失函数时，出现了一些我不明白的问题。模型正在循环训练，有时我不知道为什么或何时，模型的行为就像没有训练一样。

Epoch 1/10000
124/124 [==============================] - 6s 47ms/step - loss: 1.1125 - accuracy: 0.4600 - val_loss: 0.7640 - val_accuracy: 0.9312
Epoch 2/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.6418 - accuracy: 0.8598 - val_loss: 0.5307 - val_accuracy: 0.8718
Epoch 3/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.5434 - accuracy: 0.8768 - val_loss: 0.5416 - val_accuracy: 0.8736
Epoch 4/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.5167 - accuracy: 0.8820 - val_loss: 0.5383 - val_accuracy: 0.9165
Epoch 5/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.4948 - accuracy: 0.8898 - val_loss: 0.5136 - val_accuracy: 0.9156
Epoch 6/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.4693 - accuracy: 0.8910 - val_loss: 0.5088 - val_accuracy: 0.9130
Epoch 7/10000
124/124 [==============================] - 6s 47ms/step - loss: 0.4533 - accuracy: 0.8925 - val_loss: 0.5163 - val_accuracy: 0.8551
Epoch 8/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.4257 - accuracy: 0.8883 - val_loss: 0.5490 - val_accuracy: 0.9189
Epoch 9/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.4237 - accuracy: 0.8919 - val_loss: 0.5302 - val_accuracy: 0.8172
Epoch 10/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.4072 - accuracy: 0.8859 - val_loss: 0.5591 - val_accuracy: 0.9278
Epoch 11/10000
124/124 [==============================] - 6s 46ms/step - loss: 0.3831 - accuracy: 0.8908 - val_loss: 0.5563 - val_accuracy: 0.8937
Epoch 00011: early stopping
32695
24925
221726
5339
Epoch 1/10000
124/124 [==============================] - 6s 48ms/step - loss: 4.1699 - accuracy: 0.8661 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 2/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 3/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 4/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 5/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 6/10000
124/124 [==============================] - 6s 47ms/step - loss: 4.1813 - accuracy: 0.8664 - val_loss: 4.1812 - val_accuracy: 0.8664
Epoch 00006: early stopping

Loss, val_loss 和 accuracy 都没有下降。我认为这是损失函数中的一些问题，因为这个问题是在新的损失函数之后发生的，这个循环我可能已经完成了 10 000 次而没有错误。它可能会在 4 个周期中发生 1 个。怎么了？我将非常感谢您的帮助。谢谢

Answer 1

您的 dropout 值太低，使模型更难学习。使用更高的 dropout 值来解决您遇到的问题。

Answer 2

从建立一个隐藏层的简单模型开始，并使用流行的超参数。

您可以在移动时更新模型的同时开始微调超参数。

从我的角度来看，这是调试它的最简单和最好的方法。

如果您需要进一步的帮助，请告诉我。