Website Blog TensorPlay documentation

Automatic differentiation package - tensorplay.autograd Automatic Mixed Precision package - tensorplay.amp Distributed communication package - tensorplay.distributed Multiprocessing package - tensorplay.multiprocessing Quantization tensorplay tensorplay.audio tensorplay.compiler tensorplay.cuda tensorplay.fft tensorplay.futures tensorplay.hub tensorplay.library tensorplay.linalg tensorplay.nn tensorplay.nn.functional tensorplay.nn.init tensorplay.optim tensorplay.random tensorplay.sparse tensorplay.special tensorplay.stax tensorplay.utils.checkpoint tensorplay.utils.data tensorplay.vision Upstream anchor labels

Adadelta Adafactor Adagrad Adam Adamax AdamW ASGD AveragedModel ChainedScheduler ConstantLR CosineAnnealingLR CosineAnnealingWarmRestarts CyclicLR ExponentialLR LambdaLR LBFGS LinearLR LRScheduler MultiplicativeLR MultiStepLR Muon NAdam OneCycleLR Optimizer PolynomialLR RAdam ReduceLROnPlateau RMSprop Rprop SequentialLR SGD SparseAdam StepLR SWALR tensorplay.optim.optimizer.register_optimizer_step_post_hook tensorplay.optim.optimizer.register_optimizer_step_pre_hook tensorplay.optim.swa_utils.get_ema_avg_fn tensorplay.optim.swa_utils.get_ema_multi_avg_fn tensorplay.optim.swa_utils.get_swa_avg_fn tensorplay.optim.swa_utils.get_swa_multi_avg_fn tensorplay.optim.swa_utils.update_bn tensorplay.optim.swap_in_optimizer_params_and_state

API symbolsoptim

Copy

View Markdown Download .md

NAdam

class tensorplay.optim.nadam.NAdam(params, lr=0.002, betas=(0.9, 0.999), eps=1e-08, weight_decay=0, momentum_decay=0.004, decoupled_weight_decay=False, *, foreach=None, maximize=False, capturable=False, differentiable=False)[source]

Help improve this page

Found an error, an unclear step, or a missing example?

Was this page helpful?

Suggest an edit Report an issue

Muon

Implements Muon algorithm.

OneCycleLR

Sets the learning rate of each parameter group according to the 1cycle learning rate policy.