Keras 中的自定义 LearningRateScheduler

Question

我正在根据上一个时期的准确度实现衰减学习率。

捕获指标：

class CustomMetrics(tf.keras.callbacks.Callback):
  def on_train_begin(self, logs={}):
    self.metrics={'loss': [],'accuracy': [],'val_loss': [],'val_accuracy': []}
    self.lr=[]
  
  def on_epoch_end(self, epoch, logs={}):     
    print(f"\nEPOCH {epoch} Callng from METRICS CLASS")
    self.metrics['loss'].append(logs.get('loss'))
    self.metrics['accuracy'].append(logs.get('accuracy'))
    self.metrics['val_loss'].append(logs.get('val_loss'))
    self.metrics['val_accuracy'].append(logs.get('val_accuracy'))

自定义学习衰减：

from tensorflow.keras.callbacks import LearningRateScheduler
def changeLearningRate(epoch):
  initial_learningrate=0.1
  #print(f"EPOCH {epoch}, Calling from ChangeLearningRate:")
  lr = 0.0
  if epoch != 0:
    if custom_metrics_dict.metrics['accuracy'][epoch] < custom_metrics_dict.metrics['accuracy'][epoch-1]:    
      print(f"Accuracy @ epoch {epoch} is less than acuracy at epoch {epoch-1}")
      print("[INFO] Decreasing Learning Rate.....")
      lr = initial_learningrate*(0.1)
      print(f"LR Changed to {lr}")  
  return lr

模型准备：

input_layer = Input(shape=(2))
layer1 = Dense(32,activation='tanh',kernel_initializer=tf.random_uniform_initializer(0,1,seed=30))(input_layer)
output = Dense(2,activation='softmax',kernel_initializer=tf.random_uniform_initializer(0,1,seed=30))(layer1)


model = Model(inputs=input_layer,outputs=output)

custom_metrics_dict=CustomMetrics()
lrschedule = LearningRateScheduler(changeLearningRate, verbose=1) 

optimizer = tf.keras.optimizers.SGD(learning_rate=0.1,momentum=0.9)
model.compile(optimizer=optimizer, loss='categorical_crossentropy', metrics=['accuracy'])
model.fit(X_train,Y_train,epochs=4, validation_data=(X_test,Y_test), batch_size=16 ,callbacks=[custom_metrics_dict,lrschedule])

index out of range error 出错了。从我注意到的情况来看，每个时代，LRScheduler 代码被多次调用。我无法想出一种方法来进行适当的函数调用。接下来我可以尝试什么？

Answer 1

调度程序函数的签名是def scheduler(epoch, lr):，这意味着您应该从该参数中获取 lr。 你不应该写initial_learningrate = 0.1，如果你这样做你的lr不会衰减，当精度降低时你总是会返回相同的。 对于您检查的超出范围异常，epoch 不是 0，这意味着对于 epoch = 1，您正在检查 custom_metrics_dict.metrics['accuracy'][epoch] 和 custom_metrics_dict.metrics['accuracy'][epoch-1]，但您只存储了一个准确度值，epoch 0 没有准确度值，所以这个数组custom_metrics_dict.metrics['accuracy'] 中只有一个值

我已经用这个函数正确地运行了你的代码

from tensorflow.keras.callbacks import LearningRateScheduler
def changeLearningRate(epoch, lr):
  print(f"EPOCH {epoch},  Calling from ChangeLearningRate: {custom_metrics_dict.metrics['accuracy']}")
  if epoch > 1:
    if custom_metrics_dict.metrics['accuracy'][epoch - 1] > custom_metrics_dict.metrics['accuracy'][epoch-2]:    
      print(f"Accuracy @ epoch {epoch} is less than acuracy at epoch {epoch-1}")
      print("[INFO] Decreasing Learning Rate.....")
      lr = lr*(0.1)
      print(f"LR Changed to {lr}")  
  return lr