tensorplay.cuda

torch.cuda

`tensorplay.cuda.StreamContext`	Context-manager that selects a given stream.
`tensorplay.cuda.can_device_access_peer`	Check if peer access between two devices is possible.
`tensorplay.cuda.check_error`	Raise an error if the result of a CUDA runtime API call is not success.
`tensorplay.cuda.current_blas_handle`	Return cublasHandle_t pointer to current cuBLAS handle
`tensorplay.cuda.current_solver_handle`	Return cusolverDnHandle_t pointer to current cuSOLVER handle
`tensorplay.cuda.current_device`	Return the index of a currently selected device.
`tensorplay.cuda.current_stream`	Return the currently selected `Stream` for a given device.
`tensorplay.cuda.cudart`	Retrieves the CUDA runtime API module.
`tensorplay.cuda.default_stream`	Return the default `Stream` for a given device.
`tensorplay.cuda.device`	Context-manager that changes the selected device.
`tensorplay.cuda.device_count`	Return the number of GPUs available.
`tensorplay.cuda.device_memory_used`	Return used global (device) memory in bytes as given by nvidia-smi.
`tensorplay.cuda.device_of`	Context-manager that changes the current device to that of given object.
`tensorplay.cuda.get_arch_list`	Return list CUDA architectures this library was compiled for.
`tensorplay.cuda.get_device_capability`	Get the cuda capability of a device.
`tensorplay.cuda.get_device_name`	Get the name of a device.
`tensorplay.cuda.get_device_properties`	Get the properties of a device.
`tensorplay.cuda.get_gencode_flags`	Return NVCC gencode flags this library was compiled with.
`tensorplay.cuda.get_stream_from_external`	Return a `Stream` from an externally allocated CUDA stream.
`tensorplay.cuda.get_sync_debug_mode`	Return current value of debug mode for cuda synchronizing operations.
`tensorplay.cuda.init`	Initialize TensorPlay's CUDA state.
`tensorplay.cuda.ipc_collect`	Force collects GPU memory after it has been released by CUDA IPC.
`tensorplay.cuda.is_available`	Return a bool indicating if CUDA is currently available.
`tensorplay.cuda.is_bf16_supported`	Return a bool indicating if the current CUDA device supports dtype bfloat16.
`tensorplay.cuda.is_initialized`	Return whether TensorPlay's CUDA state has been initialized.
`tensorplay.cuda.is_tf32_supported`	Return a bool indicating if the current CUDA device supports dtype tf32.
`tensorplay.cuda.memory_usage`	Return the percent of time over the past sample period during which global (device) memory was being read or written as given by nvidia-smi.
`tensorplay.cuda.set_device`	Set the current device.
`tensorplay.cuda.set_stream`	Set the current stream. This is a wrapper API to set the stream.
`tensorplay.cuda.set_sync_debug_mode`	Set the debug mode for cuda synchronizing operations.
`tensorplay.cuda.stream`	Wrap around the Context-manager StreamContext that selects a given stream.
`tensorplay.cuda.synchronize`	Wait for all kernels in all streams on a CUDA device to complete.
`tensorplay.cuda.utilization`	Return the percent of time over the past sample period during which one or more kernels was executing on the GPU as given by nvidia-smi.
`tensorplay.cuda.temperature`	Return the average temperature of the GPU sensor in Degrees C (Centigrades).
`tensorplay.cuda.power_draw`	Return the average power draw of the GPU sensor in mW (MilliWatts) over the past sample period as given by nvidia-smi for Fermi or newer fully supported devices.
`tensorplay.cuda.clock_rate`	Return the clock speed of the GPU SM in MHz (megahertz) over the past sample period as given by nvidia-smi.
`tensorplay.cuda.AcceleratorError`
`tensorplay.cuda.OutOfMemoryError`

Random Number Generator

`tensorplay.cuda.random.get_rng_state`	Return the random number generator state of the specified GPU as a ByteTensor.
`tensorplay.cuda.random.get_rng_state_all`	Return a list of ByteTensor representing the random number states of all devices.
`tensorplay.cuda.random.set_rng_state`	Set the random number generator state of the specified GPU.
`tensorplay.cuda.random.set_rng_state_all`	Set the random number generator state of all devices.
`tensorplay.cuda.random.manual_seed`	Set the seed for generating random numbers for the current GPU.
`tensorplay.cuda.random.manual_seed_all`	Set the seed for generating random numbers on all GPUs.
`tensorplay.cuda.random.seed`	Set the seed for generating random numbers to a random number for the current GPU.
`tensorplay.cuda.random.seed_all`	Set the seed for generating random numbers to a random number on all GPUs.
`tensorplay.cuda.random.initial_seed`	Return the current random seed of the current GPU.

Streams and events

`tensorplay.cuda.streams.Stream`	Wrapper around a CUDA stream.
`tensorplay.cuda.streams.ExternalStream`	Wrapper around an externally allocated CUDA stream.
`tensorplay.cuda.streams.Event`	Wrapper around a CUDA event.

Graphs (beta)

`tensorplay.cuda.graphs.is_current_stream_capturing`	Return True if CUDA graph capture is underway on the current thread.
`tensorplay.cuda.graphs.graph_pool_handle`	Return an opaque token representing the id of a graph memory pool.
`tensorplay.cuda.graphs.CUDAGraph`	Wrapper around a CUDA graph, mirroring `torch.cuda.CUDAGraph`.
`tensorplay.cuda.graphs.graph`	Context-manager that captures CUDA work into a `tensorplay.cuda.CUDAGraph`.
`tensorplay.cuda.graphs.make_graphed_callables`	Callables that run per-iteration with CUDA graph capture (not supported).
`tensorplay.cuda.graphs.export_dot`	Export the last captured CUDA graph to a DOT file.
`tensorplay.cuda.graphs.export_graph_data`	Serialize a captured CUDA graph into a dictionary of node data.

Graph Kernel Annotations (prototype)

`tensorplay.cuda.graph_annotations.is_available`	Whether graph-kernel annotation recording is supported (always `False` here).
`tensorplay.cuda.graph_annotations.mark_kernels`	Tag the GPU work captured within its scope with user metadata.
`tensorplay.cuda.graph_annotations.get_kernel_annotations`	Return the recorded kernel annotations mapping.
`tensorplay.cuda.graph_annotations.clear_kernel_annotations`	Clear all recorded kernel annotations.

Memory management

`tensorplay.cuda.memory.caching_allocator_disabled`	Context manager that temporarily disables the CUDA caching allocator.
`tensorplay.cuda.memory.caching_allocator_enable`	Enable or disable the CUDA memory allocator.
`tensorplay.cuda.memory.use_mem_pool`	Route allocations to a MemPool (not supported by this build).
`tensorplay.cuda.nccl.version`	Returns the version of the NCCL.
`tensorplay.cuda.profiler.profile`	Enable profiling.
`tensorplay.cuda.profiler.start`	Starts cuda profiler data collection.
`tensorplay.cuda.profiler.stop`	Stops cuda profiler data collection.

NVIDIA Tools Extension (NVTX)

`tensorplay.cuda.nvtx.mark`	Describe an instantaneous event that occurred at some point.
`tensorplay.cuda.nvtx.range_push`	Push a range onto a stack of nested range span.
`tensorplay.cuda.nvtx.range_pop`	Pop a range off of a stack of nested range spans.
`tensorplay.cuda.nvtx.range`	Context manager / decorator that pushes an NVTX range at the beginning of its scope, and pops it at the end.
`tensorplay.cuda.nvtx.range_end`	Mark the end of a range for a given range_id.
`tensorplay.cuda.nvtx.range_start`	Mark the start of a range with string message.

GPUDirect Storage (prototype)

tensorplay.cuda.gds.GdsFile

A GDS file handle (not supported by this TensorPlay build).

Green Contexts (experimental)

`tensorplay.cuda.green_contexts.GreenContext`
`tensorplay.cuda.nccl.is_available`

TensorPlay-specific additions

`CUDAPluggableAllocator`	CUDA memory allocator plugin (not supported by this TensorPlay build).
`CudaError`
`DeferredCudaCallError`
`Device`
`MemPool`	MemPool context (not supported by this TensorPlay build).
`caching_allocator_alloc`	Perform a memory allocation using the CUDA memory allocator.
`caching_allocator_delete`	Delete memory allocated using the CUDA memory allocator.
`change_current_allocator`	Change the currently used memory allocator (not supported by this build).
`classproperty`
`empty_cache`	Release all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in nvidia-smi.
`get_allocator_backend`	Returns the active allocator backend as a string.
`get_per_process_memory_fraction`	Get memory fraction for a process.
`host_memory_stats`	Return a dictionary of pinned (host) allocator statistics.
`host_memory_stats_as_nested_dict`	Return the result of `host_memory_stats()` as a nested dictionary.
`is_gds_available`	Return whether GDS is available.
`list_gpu_processes`	Return a human-readable printout of the running processes and their GPU memory use for a given device.
`max_memory_allocated`	Return the maximum GPU memory occupied by tensors in bytes for a given device.
`max_memory_cached`	Deprecated; see `max_memory_reserved()`.
`max_memory_reserved`	Return the maximum GPU memory managed by the caching allocator in bytes for a given device.
`mem_get_info`	Return the global free and total GPU memory for a given device using cudaMemGetInfo.
`memory_allocated`	Return the current GPU memory occupied by tensors in bytes for a given device.
`memory_cached`	Deprecated; see `memory_reserved()`.
`memory_reserved`	Return the current GPU memory managed by the caching allocator in bytes for a given device.
`memory_snapshot`	Return a snapshot of the CUDA memory allocator state across all devices.
`memory_stats`	Return a dictionary of CUDA memory allocator statistics for a given device.
`memory_stats_as_nested_dict`	Return the result of `memory_stats()` as a nested dictionary.
`memory_summary`	Return a human-readable printout of the current memory allocator statistics for a given device.
`reset_accumulated_host_memory_stats`	Reset the "accumulated" (historical) stats tracked by the host memory allocator.
`reset_accumulated_memory_stats`	Reset the "accumulated" (historical) stats tracked by the CUDA memory allocator.
`reset_max_memory_allocated`	Reset the starting point in tracking maximum GPU memory occupied by tensors for a given device.
`reset_max_memory_cached`	Reset the starting point in tracking maximum GPU memory managed by the caching allocator for a given device.
`reset_peak_host_memory_stats`	Reset the "peak" stats tracked by the host memory allocator.
`reset_peak_memory_stats`	Reset the "peak" stats tracked by the CUDA memory allocator.
`set_per_process_memory_fraction`	Set memory fraction for a process.

tensorplay.cuda

torch.cuda

Random Number Generator

Streams and events

Graphs (beta)

Graph Kernel Annotations (prototype)

Memory management

NVIDIA Tools Extension (NVTX)

GPUDirect Storage (prototype)

Green Contexts (experimental)

TensorPlay-specific additions

Help improve this page