ModelAverager

class tensorplay.distributed.algorithms.model_averaging.ModelAverager(process_group: ProcessGroup | None = None)[source]

Base class for all model averagers.

Parameters:

process_group – The process group to be used for all-reduce. If None, the default process group, which is created by tensorplay.distributed.init_process_group(), will be used. (default: None)