latest (dev)
Copy
Latest development documentation · Updated 2026-10-08
TensorPlay documentation
TensorPlay is a tensor library for deep learning using GPUs and CPUs.
Features described in this documentation are classified by release status:
Stable (API-Stable): These features will be maintained long-term and there should generally be no major performance limitations or gaps in documentation. We also expect to maintain backwards compatibility (although breaking changes can happen and notice will be given one release ahead of time).
Unstable (API-Unstable): Encompasses all features that are under active development where APIs may change based on user feedback, requisite performance improvements or because coverage across operators is not yet complete. The APIs and performance characteristics of these features may change.
- Getting Started
- tensorplay
- Tensor attributes
- Tensor views
- Meta device
- Storage
- Type information
- Size
- Reproducibility
- DLPack
- Complex numbers
- Automatic differentiation package - tensorplay.autograd
- tensorplay.nn
- tensorplay.nn.functional
- tensorplay.nn.init
- tensorplay.nn.init.calculate_gain
- tensorplay.nn.init.uniform_
- tensorplay.nn.init.normal_
- tensorplay.nn.init.constant_
- tensorplay.nn.init.ones_
- tensorplay.nn.init.zeros_
- tensorplay.nn.init.eye_
- tensorplay.nn.init.dirac_
- tensorplay.nn.init.xavier_uniform_
- tensorplay.nn.init.xavier_normal_
- tensorplay.nn.init.kaiming_uniform_
- tensorplay.nn.init.kaiming_normal_
- tensorplay.nn.init.trunc_normal_
- tensorplay.nn.init.orthogonal_
- tensorplay.nn.init.sparse_
- tensorplay.nn.attention
- tensorplay.optim
- foreach operations
- tensorplay.cuda
- StreamContext
- tensorplay.cuda.can_device_access_peer
- tensorplay.cuda.check_error
- tensorplay.cuda.current_blas_handle
- tensorplay.cuda.current_solver_handle
- tensorplay.cuda.current_device
- tensorplay.cuda.current_stream
- tensorplay.cuda.cudart
- tensorplay.cuda.default_stream
- device
- tensorplay.cuda.device_count
- tensorplay.cuda.device_memory_used
- device_of
- tensorplay.cuda.get_arch_list
- tensorplay.cuda.get_device_capability
- tensorplay.cuda.get_device_name
- tensorplay.cuda.get_device_properties
- tensorplay.cuda.get_gencode_flags
- tensorplay.cuda.get_stream_from_external
- tensorplay.cuda.get_sync_debug_mode
- tensorplay.cuda.init
- tensorplay.cuda.ipc_collect
- tensorplay.cuda.is_available
- tensorplay.cuda.is_bf16_supported
- tensorplay.cuda.is_initialized
- tensorplay.cuda.is_tf32_supported
- tensorplay.cuda.memory_usage
- tensorplay.cuda.set_device
- tensorplay.cuda.set_stream
- tensorplay.cuda.set_sync_debug_mode
- tensorplay.cuda.stream
- tensorplay.cuda.synchronize
- tensorplay.cuda.utilization
- tensorplay.cuda.temperature
- tensorplay.cuda.power_draw
- tensorplay.cuda.clock_rate
- tensorplay.cuda.AcceleratorError
- tensorplay.cuda.OutOfMemoryError
- Random Number Generator
- Streams and events
- Graphs (beta)
- Graph Kernel Annotations (prototype)
- Memory management
- NVIDIA Tools Extension (NVTX)
- GPUDirect Storage (prototype)
- Green Contexts (experimental)
- TensorPlay-specific additions
- tensorplay.backends
- tensorplay.cuda.tunable
- tensorplay.accelerator
- Automatic Mixed Precision package - tensorplay.amp
- tensorplay.linalg
- tensorplay.fft
- tensorplay.special
- tensorplay.signal
- tensorplay.sparse
- tensorplay.random
- tensorplay.utils.data
- tensorplay.utils.checkpoint
- tensorplay.utils.checkpoint.checkpoint
- tensorplay.utils.checkpoint.checkpoint_sequential
- tensorplay.utils.checkpoint.set_checkpoint_debug_enabled
- CheckpointPolicy
- SelectiveCheckpointContext
- tensorplay.utils.checkpoint.create_selective_checkpoint_contexts
- GraphExecGroup
- tensorplay.utils.checkpoint.set_checkpoint_early_stop
- tensorplay.utils.checkpoint.set_device_states
- tensorplay.futures
- tensorplay.hub
- Multiprocessing package - tensorplay.multiprocessing
- tensorplay.library
- tensorplay.overrides
- Distributed communication package - tensorplay.distributed
- Backends
- Basics
- Initialization
- Post-Initialization
- Shutdown
- Groups
- DeviceMesh
- Point-to-point communication
- Synchronous and asynchronous collective operations
- Collective functions
- Distributed Key-Value Store
- Profiling Collective Communication
- Multi-GPU collective functions
- Object collectives
- Third-party backends
- Launch utility
- Spawn utility
- Debugging
tensorplay.distributedapplications - Logging
- Watchdog (Experimental)
- tensorplay.distributed.device_mesh
- tensorplay.distributed.tensor
- tensorplay.distributed.tensor.parallel
- tensorplay.distributed.tensor.parallel.parallelize_module
- ParallelStyle
- ColwiseParallel
- RowwiseParallel
- SequenceParallel
- PrepareModuleInput
- PrepareModuleOutput
- PrepareModuleInputOutput
- tensorplay.distributed.tensor.parallel.loss_parallel
- Parallelizing a module
- Sharding styles
- Loss with sharded logits
- Where to go next
- tensorplay.distributed.fsdp
- tensorplay.distributed.checkpoint
- tensorplay.distributed.elastic
- tensorplay.distributed.rpc
- tensorplay.distributed.pipelining
- tensorplay.distributed.optim
- tensorplay.distributed.nn
- tensorplay.distributed.autograd
- tensorplay.distributed.algorithms
- DDP Communication Hooks
- tensorplay.distributions
- tensorplay.func
- tensorplay.profiler
- tensorplay.monitor
- tensorplay.nested
- tensorplay.masked
- tensorplay.testing
- tensorplay.serialization
- tensorplay.onnx
- tensorplay.package
- tensorplay.export
- tensorplay.compiler
- Quantization - tensorplay.ao.quantization
- Pruning - tensorplay.ao.pruning
- Environment variables
- Developer Notes
- tensorplay._stax
- tensorplay.stax
- tensorplay.vision
- tensorplay.audio
- Computation-graph visualization - tensorplay.utils.viz
Indices and tables
Help improve this page
Found an error, an unclear step, or a missing example?

