Website Blog TensorPlay documentation

Automatic differentiation package - tensorplay.autograd Automatic Mixed Precision package - tensorplay.amp Distributed communication package - tensorplay.distributed Multiprocessing package - tensorplay.multiprocessing Quantization tensorplay tensorplay.audio tensorplay.compiler tensorplay.cuda tensorplay.fft tensorplay.futures tensorplay.hub tensorplay.library tensorplay.linalg tensorplay.nn tensorplay.nn.functional tensorplay.nn.init tensorplay.optim tensorplay.random tensorplay.sparse tensorplay.special tensorplay.stax tensorplay.utils.checkpoint tensorplay.utils.data tensorplay.vision Upstream anchor labels

Backend DeviceMesh FileStore P2POp ReduceOp Store TCPStore tensorplay.distributed.algorithms.model_averaging.utils.average_parameters tensorplay.distributed.algorithms.model_averaging.utils.average_parameters_or_parameter_groups tensorplay.distributed.algorithms.model_averaging.utils.get_params_to_average tensorplay.distributed.collective_utils.all_gather_object_enforce_type tensorplay.distributed.device_mesh.init_device_mesh tensorplay.distributed.distributed_c10d.all_gather tensorplay.distributed.distributed_c10d.all_gather_object tensorplay.distributed.distributed_c10d.all_reduce tensorplay.distributed.distributed_c10d.all_to_all tensorplay.distributed.distributed_c10d.all_to_all_single tensorplay.distributed.distributed_c10d.barrier tensorplay.distributed.distributed_c10d.batch_isend_irecv tensorplay.distributed.distributed_c10d.broadcast tensorplay.distributed.distributed_c10d.broadcast_object_list tensorplay.distributed.distributed_c10d.destroy_process_group tensorplay.distributed.distributed_c10d.gather tensorplay.distributed.distributed_c10d.gather_object tensorplay.distributed.distributed_c10d.get_backend tensorplay.distributed.distributed_c10d.get_global_rank tensorplay.distributed.distributed_c10d.get_group_rank tensorplay.distributed.distributed_c10d.get_process_group_ranks tensorplay.distributed.distributed_c10d.get_rank tensorplay.distributed.distributed_c10d.get_world_size tensorplay.distributed.distributed_c10d.init_process_group tensorplay.distributed.distributed_c10d.irecv tensorplay.distributed.distributed_c10d.is_available tensorplay.distributed.distributed_c10d.is_gloo_available tensorplay.distributed.distributed_c10d.is_initialized tensorplay.distributed.distributed_c10d.is_mpi_available tensorplay.distributed.distributed_c10d.is_nccl_available tensorplay.distributed.distributed_c10d.isend tensorplay.distributed.distributed_c10d.new_group tensorplay.distributed.distributed_c10d.recv tensorplay.distributed.distributed_c10d.recv_object_list tensorplay.distributed.distributed_c10d.reduce tensorplay.distributed.distributed_c10d.reduce_scatter tensorplay.distributed.distributed_c10d.scatter tensorplay.distributed.distributed_c10d.scatter_object_list tensorplay.distributed.distributed_c10d.send tensorplay.distributed.distributed_c10d.send_object_list tensorplay.distributed.launch.launch tensorplay.distributed.launch.main tensorplay.distributed.launch.parse_args tensorplay.distributed.rendezvous.register_rendezvous_handler Work

API symbolsdistributed

Copy

View Markdown Download .md

Backend

class tensorplay.distributed.distributed_c10d.Backend[source]

Help improve this page

Found an error, an unclear step, or a missing example?

Was this page helpful?

Suggest an edit Report an issue

tensorplay.diagonal

Previous Page

DeviceMesh

DeviceMesh represents a mesh of devices (torch parity).