API symbolsdistributed
Copy
tensorplay.distributed.algorithms.model_averaging.utils.get_params_to_average
- tensorplay.distributed.algorithms.model_averaging.utils.get_params_to_average(params: Iterable[Parameter] | Iterable[dict[str, Parameter]])[source]
Return a list of parameters that need to average.
This filters out the parameters that do not contain any gradients. :param params: The parameters of a model or parameter groups of an optimizer.
Help improve this page
Found an error, an unclear step, or a missing example?
Was this page helpful?
tensorplay.distributed.algorithms.model_averaging.utils.average_parameters_or_parameter_groups
Averages parameters of a model or parameter groups of an optimizer.
tensorplay.distributed.collective_utils.all_gather_object_enforce_type
Similar to plain all_gather_object but with additional type checking AFTER gather is done to ensure basic consistency. If check does not pass, all ranks will fail with exception.