如何使用 tensorflow 和 keras 加快模型训练过程

Question

我的批量大小 = 128 时期数 = 15

单个 epoch 需要 4 小时才能完成任务，因此完整的训练过程需要大量时间。就我而言，我需要提高模型训练过程的速度以保存我的体重值我该怎么做

# Training Process
results = model.fit_generator(generate_batch(orig_train, forg_train, batch_sz),
                              steps_per_epoch = num_train_samples//batch_sz,
                              epochs = 15,
                              validation_data = generate_batch(orig_val, forg_val, batch_sz),
                              validation_steps = num_val_samples//batch_sz,
                              callbacks = callbacks)

而我的回调数组定义如下，

callbacks = [
    EarlyStopping(patience=12, verbose=1),
    ReduceLROnPlateau(factor=0.1, patience=5, min_lr=0.000001, verbose=1),
    ModelCheckpoint('./Weights/model-weight-{epoch:03d}.h5', verbose=1, save_weights_only=True)
]

Answer 1

你可以做两件事：

1. 开启 XLA。

import tensorflow as tf

tf.config.optimizer.set_jit(True)

1. 开启混合精度。

from tensorflow.keras.mixed_precision import experimental as mixed_precision
policy = mixed_precision.Policy('mixed_float16')
mixed_precision.set_policy(policy)