# tensorplay.cuda

Source: https://www.tensorplay.cn/docs/cuda.html

# tensorplay.cuda

# torch.cuda

[tensorplay.cuda.StreamContext](generated/tensorplay.cuda.StreamContext.html#tensorplay.cuda.StreamContext)

Context-manager that selects a given stream.

[tensorplay.cuda.can_device_access_peer](generated/tensorplay.cuda.can_device_access_peer.html#tensorplay.cuda.can_device_access_peer)

Check if peer access between two devices is possible.

[tensorplay.cuda.check_error](generated/tensorplay.cuda.check_error.html#tensorplay.cuda.check_error)

Raise an error if the result of a CUDA runtime API call is not success.

[tensorplay.cuda.current_blas_handle](generated/tensorplay.cuda.current_blas_handle.html#tensorplay.cuda.current_blas_handle)

Return cublasHandle_t pointer to current cuBLAS handle

[tensorplay.cuda.current_solver_handle](generated/tensorplay.cuda.current_solver_handle.html#tensorplay.cuda.current_solver_handle)

Return cusolverDnHandle_t pointer to current cuSOLVER handle

[tensorplay.cuda.current_device](generated/tensorplay.cuda.current_device.html#tensorplay.cuda.current_device)

Return the index of a currently selected device.

[tensorplay.cuda.current_stream](generated/tensorplay.cuda.current_stream.html#tensorplay.cuda.current_stream)

Return the currently selected Stream for a given device.

[tensorplay.cuda.cudart](generated/tensorplay.cuda.cudart.html#tensorplay.cuda.cudart)

Retrieves the CUDA runtime API module.

[tensorplay.cuda.default_stream](generated/tensorplay.cuda.default_stream.html#tensorplay.cuda.default_stream)

Return the default Stream for a given device.

[tensorplay.cuda.device](generated/tensorplay.cuda.device.html#tensorplay.cuda.device)

Context-manager that changes the selected device.

[tensorplay.cuda.device_count](generated/tensorplay.cuda.device_count.html#tensorplay.cuda.device_count)

Return the number of GPUs available.

[tensorplay.cuda.device_memory_used](generated/tensorplay.cuda.device_memory_used.html#tensorplay.cuda.device_memory_used)

Return used global (device) memory in bytes as given by nvidia-smi.

[tensorplay.cuda.device_of](generated/tensorplay.cuda.device_of.html#tensorplay.cuda.device_of)

Context-manager that changes the current device to that of given object.

[tensorplay.cuda.get_arch_list](generated/tensorplay.cuda.get_arch_list.html#tensorplay.cuda.get_arch_list)

Return list CUDA architectures this library was compiled for.

[tensorplay.cuda.get_device_capability](generated/tensorplay.cuda.get_device_capability.html#tensorplay.cuda.get_device_capability)

Get the cuda capability of a device.

[tensorplay.cuda.get_device_name](generated/tensorplay.cuda.get_device_name.html#tensorplay.cuda.get_device_name)

Get the name of a device.

[tensorplay.cuda.get_device_properties](generated/tensorplay.cuda.get_device_properties.html#tensorplay.cuda.get_device_properties)

Get the properties of a device.

[tensorplay.cuda.get_gencode_flags](generated/tensorplay.cuda.get_gencode_flags.html#tensorplay.cuda.get_gencode_flags)

Return NVCC gencode flags this library was compiled with.

[tensorplay.cuda.get_stream_from_external](generated/tensorplay.cuda.get_stream_from_external.html#tensorplay.cuda.get_stream_from_external)

Return a Stream from an externally allocated CUDA stream.

[tensorplay.cuda.get_sync_debug_mode](generated/tensorplay.cuda.get_sync_debug_mode.html#tensorplay.cuda.get_sync_debug_mode)

Return current value of debug mode for cuda synchronizing operations.

[tensorplay.cuda.init](generated/tensorplay.cuda.init.html#tensorplay.cuda.init)

Initialize TensorPlay's CUDA state.

[tensorplay.cuda.ipc_collect](generated/tensorplay.cuda.ipc_collect.html#tensorplay.cuda.ipc_collect)

Force collects GPU memory after it has been released by CUDA IPC.

[tensorplay.cuda.is_available](generated/tensorplay.cuda.is_available.html#tensorplay.cuda.is_available)

Return a bool indicating if CUDA is currently available.

[tensorplay.cuda.is_bf16_supported](generated/tensorplay.cuda.is_bf16_supported.html#tensorplay.cuda.is_bf16_supported)

Return a bool indicating if the current CUDA device supports dtype bfloat16.

[tensorplay.cuda.is_initialized](generated/tensorplay.cuda.is_initialized.html#tensorplay.cuda.is_initialized)

Return whether TensorPlay's CUDA state has been initialized.

[tensorplay.cuda.is_tf32_supported](generated/tensorplay.cuda.is_tf32_supported.html#tensorplay.cuda.is_tf32_supported)

Return a bool indicating if the current CUDA device supports dtype tf32.

[tensorplay.cuda.memory_usage](generated/tensorplay.cuda.memory_usage.html#tensorplay.cuda.memory_usage)

Return the percent of time over the past sample period during which global (device) memory was being read or written as given by nvidia-smi.

[tensorplay.cuda.set_device](generated/tensorplay.cuda.set_device.html#tensorplay.cuda.set_device)

Set the current device.

[tensorplay.cuda.set_stream](generated/tensorplay.cuda.set_stream.html#tensorplay.cuda.set_stream)

Set the current stream. This is a wrapper API to set the stream.

[tensorplay.cuda.set_sync_debug_mode](generated/tensorplay.cuda.set_sync_debug_mode.html#tensorplay.cuda.set_sync_debug_mode)

Set the debug mode for cuda synchronizing operations.

[tensorplay.cuda.stream](generated/tensorplay.cuda.stream.html#tensorplay.cuda.stream)

Wrap around the Context-manager StreamContext that selects a given stream.

[tensorplay.cuda.synchronize](generated/tensorplay.cuda.synchronize.html#tensorplay.cuda.synchronize)

Wait for all kernels in all streams on a CUDA device to complete.

[tensorplay.cuda.utilization](generated/tensorplay.cuda.utilization.html#tensorplay.cuda.utilization)

Return the percent of time over the past sample period during which one or more kernels was executing on the GPU as given by nvidia-smi.

[tensorplay.cuda.temperature](generated/tensorplay.cuda.temperature.html#tensorplay.cuda.temperature)

Return the average temperature of the GPU sensor in Degrees C (Centigrades).

[tensorplay.cuda.power_draw](generated/tensorplay.cuda.power_draw.html#tensorplay.cuda.power_draw)

Return the average power draw of the GPU sensor in mW (MilliWatts) over the past sample period as given by nvidia-smi for Fermi or newer fully supported devices.

[tensorplay.cuda.clock_rate](generated/tensorplay.cuda.clock_rate.html#tensorplay.cuda.clock_rate)

Return the clock speed of the GPU SM in MHz (megahertz) over the past sample period as given by nvidia-smi.

[tensorplay.cuda.AcceleratorError](generated/tensorplay.cuda.AcceleratorError.html#tensorplay.cuda.AcceleratorError)

[tensorplay.cuda.OutOfMemoryError](generated/tensorplay.cuda.OutOfMemoryError.html#tensorplay.cuda.OutOfMemoryError)

## Random Number Generator

[tensorplay.cuda.random.get_rng_state](generated/tensorplay.cuda.random.get_rng_state.html#tensorplay.cuda.random.get_rng_state)

Return the random number generator state of the specified GPU as a ByteTensor.

[tensorplay.cuda.random.get_rng_state_all](generated/tensorplay.cuda.random.get_rng_state_all.html#tensorplay.cuda.random.get_rng_state_all)

Return a list of ByteTensor representing the random number states of all devices.

[tensorplay.cuda.random.set_rng_state](generated/tensorplay.cuda.random.set_rng_state.html#tensorplay.cuda.random.set_rng_state)

Set the random number generator state of the specified GPU.

[tensorplay.cuda.random.set_rng_state_all](generated/tensorplay.cuda.random.set_rng_state_all.html#tensorplay.cuda.random.set_rng_state_all)

Set the random number generator state of all devices.

[tensorplay.cuda.random.manual_seed](generated/tensorplay.cuda.random.manual_seed.html#tensorplay.cuda.random.manual_seed)

Set the seed for generating random numbers for the current GPU.

[tensorplay.cuda.random.manual_seed_all](generated/tensorplay.cuda.random.manual_seed_all.html#tensorplay.cuda.random.manual_seed_all)

Set the seed for generating random numbers on all GPUs.

[tensorplay.cuda.random.seed](generated/tensorplay.cuda.random.seed.html#tensorplay.cuda.random.seed)

Set the seed for generating random numbers to a random number for the current GPU.

[tensorplay.cuda.random.seed_all](generated/tensorplay.cuda.random.seed_all.html#tensorplay.cuda.random.seed_all)

Set the seed for generating random numbers to a random number on all GPUs.

[tensorplay.cuda.random.initial_seed](generated/tensorplay.cuda.random.initial_seed.html#tensorplay.cuda.random.initial_seed)

Return the current random seed of the current GPU.

## Streams and events

[tensorplay.cuda.streams.Stream](generated/tensorplay.cuda.streams.Stream.html#tensorplay.cuda.streams.Stream)

Wrapper around a CUDA stream.

[tensorplay.cuda.streams.ExternalStream](generated/tensorplay.cuda.streams.ExternalStream.html#tensorplay.cuda.streams.ExternalStream)

Wrapper around an externally allocated CUDA stream.

[tensorplay.cuda.streams.Event](generated/tensorplay.cuda.streams.Event.html#tensorplay.cuda.streams.Event)

Wrapper around a CUDA event.

## Graphs (beta)

[tensorplay.cuda.graphs.is_current_stream_capturing](generated/tensorplay.cuda.graphs.is_current_stream_capturing.html#tensorplay.cuda.graphs.is_current_stream_capturing)

Return True if CUDA graph capture is underway on the current thread.

[tensorplay.cuda.graphs.graph_pool_handle](generated/tensorplay.cuda.graphs.graph_pool_handle.html#tensorplay.cuda.graphs.graph_pool_handle)

Return an opaque token representing the id of a graph memory pool.

[tensorplay.cuda.graphs.CUDAGraph](generated/tensorplay.cuda.graphs.CUDAGraph.html#tensorplay.cuda.graphs.CUDAGraph)

Wrapper around a CUDA graph, mirroring torch.cuda.CUDAGraph.

[tensorplay.cuda.graphs.graph](generated/tensorplay.cuda.graphs.graph.html#tensorplay.cuda.graphs.graph)

Context-manager that captures CUDA work into a tensorplay.cuda.CUDAGraph.

[tensorplay.cuda.graphs.make_graphed_callables](generated/tensorplay.cuda.graphs.make_graphed_callables.html#tensorplay.cuda.graphs.make_graphed_callables)

Callables that run per-iteration with CUDA graph capture (not supported).

[tensorplay.cuda.graphs.export_dot](generated/tensorplay.cuda.graphs.export_dot.html#tensorplay.cuda.graphs.export_dot)

Export the last captured CUDA graph to a DOT file.

[tensorplay.cuda.graphs.export_graph_data](generated/tensorplay.cuda.graphs.export_graph_data.html#tensorplay.cuda.graphs.export_graph_data)

Serialize a captured CUDA graph into a dictionary of node data.

## Graph Kernel Annotations (prototype)

[tensorplay.cuda.graph_annotations.is_available](generated/tensorplay.cuda.graph_annotations.is_available.html#tensorplay.cuda.graph_annotations.is_available)

Whether graph-kernel annotation recording is supported (always False here).

[tensorplay.cuda.graph_annotations.mark_kernels](generated/tensorplay.cuda.graph_annotations.mark_kernels.html#tensorplay.cuda.graph_annotations.mark_kernels)

Tag the GPU work captured within its scope with user metadata.

[tensorplay.cuda.graph_annotations.get_kernel_annotations](generated/tensorplay.cuda.graph_annotations.get_kernel_annotations.html#tensorplay.cuda.graph_annotations.get_kernel_annotations)

Return the recorded kernel annotations mapping.

[tensorplay.cuda.graph_annotations.clear_kernel_annotations](generated/tensorplay.cuda.graph_annotations.clear_kernel_annotations.html#tensorplay.cuda.graph_annotations.clear_kernel_annotations)

Clear all recorded kernel annotations.

## Memory management

[tensorplay.cuda.memory.caching_allocator_disabled](generated/tensorplay.cuda.memory.caching_allocator_disabled.html#tensorplay.cuda.memory.caching_allocator_disabled)

Context manager that temporarily disables the CUDA caching allocator.

[tensorplay.cuda.memory.caching_allocator_enable](generated/tensorplay.cuda.memory.caching_allocator_enable.html#tensorplay.cuda.memory.caching_allocator_enable)

Enable or disable the CUDA memory allocator.

[tensorplay.cuda.memory.use_mem_pool](generated/tensorplay.cuda.memory.use_mem_pool.html#tensorplay.cuda.memory.use_mem_pool)

Route allocations to a MemPool (not supported by this build).

[tensorplay.cuda.nccl.version](generated/tensorplay.cuda.nccl.version.html#tensorplay.cuda.nccl.version)

Returns the version of the NCCL.

[tensorplay.cuda.profiler.profile](generated/tensorplay.cuda.profiler.profile.html#tensorplay.cuda.profiler.profile)

Enable profiling.

[tensorplay.cuda.profiler.start](generated/tensorplay.cuda.profiler.start.html#tensorplay.cuda.profiler.start)

Starts cuda profiler data collection.

[tensorplay.cuda.profiler.stop](generated/tensorplay.cuda.profiler.stop.html#tensorplay.cuda.profiler.stop)

Stops cuda profiler data collection.

## NVIDIA Tools Extension (NVTX)

[tensorplay.cuda.nvtx.mark](generated/tensorplay.cuda.nvtx.mark.html#tensorplay.cuda.nvtx.mark)

Describe an instantaneous event that occurred at some point.

[tensorplay.cuda.nvtx.range_push](generated/tensorplay.cuda.nvtx.range_push.html#tensorplay.cuda.nvtx.range_push)

Push a range onto a stack of nested range span.

[tensorplay.cuda.nvtx.range_pop](generated/tensorplay.cuda.nvtx.range_pop.html#tensorplay.cuda.nvtx.range_pop)

Pop a range off of a stack of nested range spans.

[tensorplay.cuda.nvtx.range](generated/tensorplay.cuda.nvtx.range.html#tensorplay.cuda.nvtx.range)

Context manager / decorator that pushes an NVTX range at the beginning of its scope, and pops it at the end.

[tensorplay.cuda.nvtx.range_end](generated/tensorplay.cuda.nvtx.range_end.html#tensorplay.cuda.nvtx.range_end)

Mark the end of a range for a given range_id.

[tensorplay.cuda.nvtx.range_start](generated/tensorplay.cuda.nvtx.range_start.html#tensorplay.cuda.nvtx.range_start)

Mark the start of a range with string message.

## GPUDirect Storage (prototype)

[tensorplay.cuda.gds.GdsFile](generated/tensorplay.cuda.gds.GdsFile.html#tensorplay.cuda.gds.GdsFile)

A GDS file handle (not supported by this TensorPlay build).

## Green Contexts (experimental)

[tensorplay.cuda.green_contexts.GreenContext](generated/tensorplay.cuda.green_contexts.GreenContext.html#tensorplay.cuda.green_contexts.GreenContext)

[tensorplay.cuda.nccl.is_available](generated/tensorplay.cuda.nccl.is_available.html#tensorplay.cuda.nccl.is_available)

## TensorPlay-specific additions

[CUDAPluggableAllocator](generated/tensorplay.cuda.CUDAPluggableAllocator.html#tensorplay.cuda.CUDAPluggableAllocator)

CUDA memory allocator plugin (not supported by this TensorPlay build).

[CudaError](generated/tensorplay.cuda.CudaError.html#tensorplay.cuda.CudaError)

[DeferredCudaCallError](generated/tensorplay.cuda.DeferredCudaCallError.html#tensorplay.cuda.DeferredCudaCallError)

[Device](generated/tensorplay.cuda.Device.html#tensorplay.cuda.Device)

[MemPool](generated/tensorplay.cuda.MemPool.html#tensorplay.cuda.MemPool)

MemPool context (not supported by this TensorPlay build).

[caching_allocator_alloc](generated/tensorplay.cuda.caching_allocator_alloc.html#tensorplay.cuda.caching_allocator_alloc)

Perform a memory allocation using the CUDA memory allocator.

[caching_allocator_delete](generated/tensorplay.cuda.caching_allocator_delete.html#tensorplay.cuda.caching_allocator_delete)

Delete memory allocated using the CUDA memory allocator.

[change_current_allocator](generated/tensorplay.cuda.change_current_allocator.html#tensorplay.cuda.change_current_allocator)

Change the currently used memory allocator (not supported by this build).

[classproperty](generated/tensorplay.cuda.classproperty.html#tensorplay.cuda.classproperty)

[empty_cache](generated/tensorplay.cuda.empty_cache.html#tensorplay.cuda.empty_cache)

Release all unoccupied cached memory currently held by the caching allocator so that those can be used in other GPU application and visible in nvidia-smi.

[get_allocator_backend](generated/tensorplay.cuda.get_allocator_backend.html#tensorplay.cuda.get_allocator_backend)

Returns the active allocator backend as a string.

[get_per_process_memory_fraction](generated/tensorplay.cuda.get_per_process_memory_fraction.html#tensorplay.cuda.get_per_process_memory_fraction)

Get memory fraction for a process.

[host_memory_stats](generated/tensorplay.cuda.host_memory_stats.html#tensorplay.cuda.host_memory_stats)

Return a dictionary of pinned (host) allocator statistics.

[host_memory_stats_as_nested_dict](generated/tensorplay.cuda.host_memory_stats_as_nested_dict.html#tensorplay.cuda.host_memory_stats_as_nested_dict)

Return the result of [host_memory_stats()](generated/tensorplay.cuda.host_memory_stats.html#tensorplay.cuda.host_memory_stats) as a nested dictionary.

[is_gds_available](generated/tensorplay.cuda.is_gds_available.html#tensorplay.cuda.is_gds_available)

Return whether GDS is available.

[list_gpu_processes](generated/tensorplay.cuda.list_gpu_processes.html#tensorplay.cuda.list_gpu_processes)

Return a human-readable printout of the running processes and their GPU memory use for a given device.

[max_memory_allocated](generated/tensorplay.cuda.max_memory_allocated.html#tensorplay.cuda.max_memory_allocated)

Return the maximum GPU memory occupied by tensors in bytes for a given device.

[max_memory_cached](generated/tensorplay.cuda.max_memory_cached.html#tensorplay.cuda.max_memory_cached)

Deprecated; see [max_memory_reserved()](generated/tensorplay.cuda.max_memory_reserved.html#tensorplay.cuda.max_memory_reserved).

[max_memory_reserved](generated/tensorplay.cuda.max_memory_reserved.html#tensorplay.cuda.max_memory_reserved)

Return the maximum GPU memory managed by the caching allocator in bytes for a given device.

[mem_get_info](generated/tensorplay.cuda.mem_get_info.html#tensorplay.cuda.mem_get_info)

Return the global free and total GPU memory for a given device using cudaMemGetInfo.

[memory_allocated](generated/tensorplay.cuda.memory_allocated.html#tensorplay.cuda.memory_allocated)

Return the current GPU memory occupied by tensors in bytes for a given device.

[memory_cached](generated/tensorplay.cuda.memory_cached.html#tensorplay.cuda.memory_cached)

Deprecated; see [memory_reserved()](generated/tensorplay.cuda.memory_reserved.html#tensorplay.cuda.memory_reserved).

[memory_reserved](generated/tensorplay.cuda.memory_reserved.html#tensorplay.cuda.memory_reserved)

Return the current GPU memory managed by the caching allocator in bytes for a given device.

[memory_snapshot](generated/tensorplay.cuda.memory_snapshot.html#tensorplay.cuda.memory_snapshot)

Return a snapshot of the CUDA memory allocator state across all devices.

[memory_stats](generated/tensorplay.cuda.memory_stats.html#tensorplay.cuda.memory_stats)

Return a dictionary of CUDA memory allocator statistics for a given device.

[memory_stats_as_nested_dict](generated/tensorplay.cuda.memory_stats_as_nested_dict.html#tensorplay.cuda.memory_stats_as_nested_dict)

Return the result of [memory_stats()](generated/tensorplay.cuda.memory_stats.html#tensorplay.cuda.memory_stats) as a nested dictionary.

[memory_summary](generated/tensorplay.cuda.memory_summary.html#tensorplay.cuda.memory_summary)

Return a human-readable printout of the current memory allocator statistics for a given device.

[reset_accumulated_host_memory_stats](generated/tensorplay.cuda.reset_accumulated_host_memory_stats.html#tensorplay.cuda.reset_accumulated_host_memory_stats)

Reset the "accumulated" (historical) stats tracked by the host memory allocator.

[reset_accumulated_memory_stats](generated/tensorplay.cuda.reset_accumulated_memory_stats.html#tensorplay.cuda.reset_accumulated_memory_stats)

Reset the "accumulated" (historical) stats tracked by the CUDA memory allocator.

[reset_max_memory_allocated](generated/tensorplay.cuda.reset_max_memory_allocated.html#tensorplay.cuda.reset_max_memory_allocated)

Reset the starting point in tracking maximum GPU memory occupied by tensors for a given device.

[reset_max_memory_cached](generated/tensorplay.cuda.reset_max_memory_cached.html#tensorplay.cuda.reset_max_memory_cached)

Reset the starting point in tracking maximum GPU memory managed by the caching allocator for a given device.

[reset_peak_host_memory_stats](generated/tensorplay.cuda.reset_peak_host_memory_stats.html#tensorplay.cuda.reset_peak_host_memory_stats)

Reset the "peak" stats tracked by the host memory allocator.

[reset_peak_memory_stats](generated/tensorplay.cuda.reset_peak_memory_stats.html#tensorplay.cuda.reset_peak_memory_stats)

Reset the "peak" stats tracked by the CUDA memory allocator.

[set_per_process_memory_fraction](generated/tensorplay.cuda.set_per_process_memory_fraction.html#tensorplay.cuda.set_per_process_memory_fraction)

Set memory fraction for a process.