Website Blog TensorPlay documentation

tensorplay API tensorplay._composite_funcs API tensorplay._einsum API tensorplay._shape_funcs API tensorplay._stax.api API tensorplay.accelerator API tensorplay.amp.autocast_mode API tensorplay.ao.nn.intrinsic API tensorplay.ao.nn.qat API tensorplay.ao.nn.quantized API tensorplay.ao.nn.quantized.dynamic API tensorplay.ao.pruning API tensorplay.ao.quantization API tensorplay.autograd API tensorplay.autograd.anomaly_mode API tensorplay.autograd.forward_ad API tensorplay.autograd.function API tensorplay.autograd.functional API tensorplay.autograd.grad_mode API tensorplay.autograd.gradcheck API tensorplay.autograd.graph API tensorplay.autograd.profiler_util API tensorplay.autograd.variable API tensorplay.backends.cpu API tensorplay.backends.cuda API tensorplay.backends.mkl API tensorplay.backends.nnpack API tensorplay.backends.openmp API tensorplay.compiler API tensorplay.cuda API tensorplay.cuda.gds API tensorplay.cuda.graph_annotations API tensorplay.cuda.graphs API tensorplay.cuda.green_contexts API tensorplay.cuda.memory API tensorplay.cuda.nccl API tensorplay.cuda.nvtx API tensorplay.cuda.profiler API tensorplay.cuda.random API tensorplay.cuda.streams API tensorplay.cuda.tunable API tensorplay.distributed.algorithms API tensorplay.distributed.algorithms.ddp_comm_hooks.ddp_zero_hook API tensorplay.distributed.algorithms.ddp_comm_hooks.default_hooks API tensorplay.distributed.algorithms.ddp_comm_hooks.post_localSGD_hook API tensorplay.distributed.algorithms.ddp_comm_hooks.powerSGD_hook API tensorplay.distributed.algorithms.ddp_comm_hooks.quantization_hooks API tensorplay.distributed.algorithms.model_averaging API tensorplay.distributed.autograd API tensorplay.distributed.checkpoint API tensorplay.distributed.device_mesh API tensorplay.distributed.elastic.agent.server API tensorplay.distributed.elastic.events API tensorplay.distributed.elastic.metrics API tensorplay.distributed.elastic.multiprocessing API tensorplay.distributed.elastic.rendezvous API tensorplay.distributed.elastic.timer API tensorplay.distributed.elastic.utils API tensorplay.distributed.elastic.utils.data API tensorplay.distributed.fsdp API tensorplay.distributed.fsdp.sharded_grad_scaler API tensorplay.distributed.launcher API tensorplay.distributed.nn API tensorplay.distributed.nn.api API tensorplay.distributed.nn.functional API tensorplay.distributed.optim API tensorplay.distributed.pipelining API tensorplay.distributed.rpc API tensorplay.distributed.tensor API tensorplay.distributed.tensor.parallel API tensorplay.distributions API tensorplay.export API tensorplay.fft._frequency API tensorplay.fft._transforms API tensorplay.func API tensorplay.functional API tensorplay.futures API tensorplay.library API tensorplay.linalg API tensorplay.masked API tensorplay.monitor API tensorplay.multiprocessing API tensorplay.nested API tensorplay.nn.attention API tensorplay.nn.attention.bias API tensorplay.nn.attention.varlen API tensorplay.nn.functional API tensorplay.nn.init API tensorplay.nn.modules.flatten API tensorplay.nn.modules.lazy API tensorplay.nn.modules.module API tensorplay.nn.parameter API tensorplay.nn.utils.rnn API tensorplay.nn.utils.stateless API tensorplay.onnx API tensorplay.overrides API tensorplay.package API tensorplay.profiler API tensorplay.profiler.profiler API tensorplay.random API tensorplay.serialization API tensorplay.serialization.api API tensorplay.signal.windows API tensorplay.special._elementwise API tensorplay.special._error API tensorplay.special._gamma API tensorplay.stax API tensorplay.testing API tensorplay.utils.checkpoint API tensorplay.utils.data API tensorplay.utils.viz API

Automatic differentiation package - tensorplay.autograd Automatic Mixed Precision package - tensorplay.amp Complex numbers Computation-graph visualization - tensorplay.utils.viz DDP Communication Hooks Distributed communication package - tensorplay.distributed DLPack Environment variables foreach operations Index Meta device Multiprocessing package - tensorplay.multiprocessing Patching batch norm Pruning - tensorplay.ao.pruning Python Module Index Quantization - tensorplay.ao.quantization Reproducibility Search Size Storage Tensor attributes Tensor views tensorplay tensorplay._stax tensorplay.accelerator tensorplay.audio tensorplay.backends tensorplay.compiler tensorplay.cuda tensorplay.cuda.tunable tensorplay.distributed.algorithms tensorplay.distributed.autograd tensorplay.distributed.checkpoint tensorplay.distributed.device_mesh tensorplay.distributed.elastic tensorplay.distributed.fsdp tensorplay.distributed.fsdp.fully_shard tensorplay.distributed.nn tensorplay.distributed.optim tensorplay.distributed.pipelining tensorplay.distributed.rpc tensorplay.distributed.tensor tensorplay.distributed.tensor.parallel tensorplay.distributions tensorplay.export tensorplay.fft tensorplay.func tensorplay.func API reference tensorplay.func whirlwind tour tensorplay.futures tensorplay.hub tensorplay.library tensorplay.linalg tensorplay.masked tensorplay.monitor tensorplay.nested tensorplay.nn tensorplay.nn.attention tensorplay.nn.attention.bias tensorplay.nn.attention.omni_attention tensorplay.nn.attention.varlen tensorplay.nn.functional tensorplay.nn.init tensorplay.onnx tensorplay.optim tensorplay.overrides tensorplay.package tensorplay.profiler tensorplay.random tensorplay.serialization tensorplay.signal tensorplay.sparse tensorplay.special tensorplay.stax tensorplay.testing tensorplay.utils.checkpoint tensorplay.utils.data tensorplay.vision Type information UX limitations

API reference

Documentation/API reference

latest (dev)

latest (dev)2026-10-08

Copy

Latest development documentation · Updated 2026-10-08

tensorplay.distributed.nn.functional API

Functions 10

all_gather_single

functionFull reference ↗

tensorplay.distributed.nn.functional.all_gather_single(output_tensor, input_tensor, group=None)[source]

all_gather

functionFull reference ↗

tensorplay.distributed.nn.functional.all_gather(tensor: Tensor, group=None)[source]

all_reduce

functionFull reference ↗

tensorplay.distributed.nn.functional.all_reduce(tensor: Tensor, op: int = 0, group=None)[source]

all_to_all_single

functionFull reference ↗

tensorplay.distributed.nn.functional.all_to_all_single(output, input, output_split_sizes=None, input_split_sizes=None, group=None)[source]

all_to_all

functionFull reference ↗

tensorplay.distributed.nn.functional.all_to_all(output_tensor_list, input_tensor_list, group=None)[source]

broadcast

functionFull reference ↗

tensorplay.distributed.nn.functional.broadcast(tensor: Tensor, src: int, group=None)[source]

gather

functionFull reference ↗

tensorplay.distributed.nn.functional.gather(tensor: Tensor, dst: int = 0, group=None)[source]

reduce_scatter

functionFull reference ↗

tensorplay.distributed.nn.functional.reduce_scatter(output, input_list, op: int = 0, group=None)[source]

reduce

functionFull reference ↗

tensorplay.distributed.nn.functional.reduce(tensor: Tensor, dst: int, op: int = 0, group=None)[source]

scatter

functionFull reference ↗

tensorplay.distributed.nn.functional.scatter(tensors, src: int = 0, group=None)[source]

Help improve this page

Found an error, an unclear step, or a missing example?

Was this page helpful?

Suggest an edit Report an issue

tensorplay.distributed.nn.api API

Complete API reference for tensorplay.distributed.nn.api, including signatures, parameters, examples and members.

tensorplay.distributed.optim API

Complete API reference for tensorplay.distributed.optim, including signatures, parameters, examples and members.

On this page

Functions 10 all_gather_single all_gather all_reduce all_to_all_single all_to_all broadcast gather reduce_scatter reduce scatter