这篇教程Keras RAdam写得很实用，希望能帮到您。
Keras RAdam
    
    
[中文|English]
Unofficial implementation of RAdam in Keras and TensorFlow.
Install
pip install keras-rectified-adam
External Link
Usage
import keras
import numpy as np
from keras_radam import RAdam
# Build toy model with RAdam optimizer
model = keras.models.Sequential()
model.add(keras.layers.Dense(input_shape=(17,), units=3))
model.compile(RAdam(), loss='mse')
# Generate toy data
x = np.random.standard_normal((4096 * 30, 17))
w = np.random.standard_normal((17, 3))
y = np.dot(x, w)
# Fit
model.fit(x, y, epochs=5)
TensorFlow without Keras
from keras_radam.training import RAdamOptimizer
RAdamOptimizer(learning_rate=1e-3)
Use Warmup
from keras_radam import RAdam
RAdam(total_steps=10000, warmup_proportion=0.1, min_lr=1e-5)
Q & A
深度学习-神经网络模型参数选择
keras各种优化器优化器的用法 |
