TensorPlay documentation

TensorPlay is a PyTorch-compatible tensor library for deep learning using GPUs and CPUs.

Features described in this documentation are classified by release status:

Stable (API-Stable): These features will be maintained long-term and there should generally be no major performance limitations or gaps in documentation. We also expect to maintain backwards compatibility (although breaking changes can happen and notice will be given one release ahead of time).

Unstable (API-Unstable): Encompasses all features that are under active development where APIs may change based on user feedback, requisite performance improvements or because coverage across operators is not yet complete. The APIs and performance characteristics of these features may change.

tensorplay
- Tensors
- Generators
- Random sampling
- Serialization
- Parallelism
- Locally disabling gradient computation
- Utilities
- Optimizations
- TensorPlay-specific additions
Automatic differentiation package - tensorplay.autograd
Automatic differentiation package - torch.autograd
- tensorplay._C._autograd.backward
- tensorplay.autograd.grad
- Functional higher level API
- Function
- Custom Function utilities
- Numerical gradient checking
- Debugging and anomaly detection
tensorplay.nn
- Containers
- Convolution Layers
- Pooling layers
- Padding Layers
- Non-linear Activations (weighted sum, nonlinearity)
- Non-linear Activations (other)
- Normalization Layers
- Recurrent Layers
- Transformer Layers
- Linear Layers
- Dropout Layers
- Sparse Layers
- Distance Functions
- Loss Functions
- Vision Layers
- Shuffle Layers
- DataParallel Layers (multi-GPU, distributed)
- Utilities
- Lazy Modules Initialization
- TensorPlay-specific additions
tensorplay.nn.functional
- Convolution functions
- Pooling functions
- Attention Mechanisms
- Non-linear activation functions
- Linear functions
- Dropout functions
- Sparse functions
- Distance functions
- Loss functions
- Vision functions
- Low-Precision functions
- TensorPlay-specific additions
tensorplay.nn.init
torch.nn.init
- tensorplay.nn.init.calculate_gain
- tensorplay.nn.init.uniform_
- tensorplay.nn.init.normal_
- tensorplay.nn.init.constant_
- tensorplay.nn.init.ones_
- tensorplay.nn.init.zeros_
- tensorplay.nn.init.eye_
- tensorplay.nn.init.dirac_
- tensorplay.nn.init.xavier_uniform_
- tensorplay.nn.init.xavier_normal_
- tensorplay.nn.init.kaiming_uniform_
- tensorplay.nn.init.kaiming_normal_
- tensorplay.nn.init.trunc_normal_
- tensorplay.nn.init.orthogonal_
- tensorplay.nn.init.sparse_
tensorplay.optim
- Base class
- Module-level hooks
- Utilities
- Algorithms
- How to adjust learning rate
tensorplay.cuda
torch.cuda
- StreamContext
- tensorplay.cuda.can_device_access_peer
- tensorplay.cuda.check_error
- tensorplay.cuda.current_blas_handle
- tensorplay.cuda.current_solver_handle
- tensorplay.cuda.current_device
- tensorplay.cuda.current_stream
- tensorplay.cuda.cudart
- tensorplay.cuda.default_stream
- device
- tensorplay.cuda.device_count
- tensorplay.cuda.device_memory_used
- device_of
- tensorplay.cuda.get_arch_list
- tensorplay.cuda.get_device_capability
- tensorplay.cuda.get_device_name
- tensorplay.cuda.get_device_properties
- tensorplay.cuda.get_gencode_flags
- tensorplay.cuda.get_stream_from_external
- tensorplay.cuda.get_sync_debug_mode
- tensorplay.cuda.init
- tensorplay.cuda.ipc_collect
- tensorplay.cuda.is_available
- tensorplay.cuda.is_bf16_supported
- tensorplay.cuda.is_initialized
- tensorplay.cuda.is_tf32_supported
- tensorplay.cuda.memory_usage
- tensorplay.cuda.set_device
- tensorplay.cuda.set_stream
- tensorplay.cuda.set_sync_debug_mode
- tensorplay.cuda.stream
- tensorplay.cuda.synchronize
- tensorplay.cuda.utilization
- tensorplay.cuda.temperature
- tensorplay.cuda.power_draw
- tensorplay.cuda.clock_rate
- tensorplay.cuda.AcceleratorError
- tensorplay.cuda.OutOfMemoryError
- Random Number Generator
- Streams and events
- Graphs (beta)
- Graph Kernel Annotations (prototype)
- Memory management
- NVIDIA Tools Extension (NVTX)
- GPUDirect Storage (prototype)
- Green Contexts (experimental)
- TensorPlay-specific additions
Automatic Mixed Precision package - tensorplay.amp
- Autocasting
- Gradient Scaling
tensorplay.linalg
- Matrix Properties
- Decompositions
- Solvers
- Inverses
- Matrix Functions
- Matrix Products
- Tensor Operations
- Misc
- Experimental Functions
tensorplay.fft
- Fast Fourier Transforms
- Helper Functions
tensorplay.special
- Functions
tensorplay.sparse
- Tensor methods and sparse
tensorplay.random
- Random Generator
tensorplay.utils.data
- Memory Pinning
tensorplay.utils.checkpoint
torch.utils.checkpoint
- tensorplay.utils.checkpoint.checkpoint
- tensorplay.utils.checkpoint.checkpoint_sequential
- CheckpointPolicy
- tensorplay.utils.checkpoint.set_checkpoint_early_stop
tensorplay.futures
torch.futures
- Future
tensorplay.hub
- Loading models from Hub
Multiprocessing package - tensorplay.multiprocessing
- API Reference
tensorplay.library
- Choosing the kind of custom op
- Extending custom ops (created from Python or C++)
- Low-level APIs
Distributed communication package - tensorplay.distributed
Distributed communication package - torch.distributed
- tensorplay.distributed.rendezvous.register_rendezvous_handler
- tensorplay.distributed.algorithms.model_averaging.utils.average_parameters
- tensorplay.distributed.algorithms.model_averaging.utils.average_parameters_or_parameter_groups
- tensorplay.distributed.algorithms.model_averaging.utils.get_params_to_average
- Initialization
- Post-Initialization
- Groups
- DeviceMesh
- Point-to-point communication
- Collective functions
- Distributed Key-Value Store
- Launch utility
- Watchdog (Experimental)
Quantization
- Quantization API Reference (Kept since APIs are still public)
tensorplay.compiler
- Graph IR
- CUDA graphs
- Caching and errors
- Related modules
tensorplay.stax
- Functions
tensorplay.vision
- Top-level functions
- Submodules
tensorplay.audio
- Top-level functions
- Submodules

Indices and tables

Index
Module Index
Search Page

TensorPlay documentation

Indices and tables

Help improve this page