tensorplay.nn

Containers

`tensorplay.nn.modules.module.Module`	Base class for all neural network modules.
`tensorplay.nn.modules.container.Sequential`	A sequential container.
`tensorplay.nn.modules.container.ModuleList`	Holds submodules in a list.
`tensorplay.nn.modules.container.ModuleDict`	Holds submodules in a dictionary.
`tensorplay.nn.modules.container.ParameterList`	Holds parameters in a list.
`tensorplay.nn.modules.container.ParameterDict`	Holds parameters in a dictionary.
`tensorplay.nn.modules.module.register_module_forward_pre_hook`	Register a forward pre-hook common to all modules.
`tensorplay.nn.modules.module.register_module_forward_hook`	Register a global forward hook for all the modules.
`tensorplay.nn.modules.module.register_module_backward_hook`	Register a backward hook common to all the modules.
`tensorplay.nn.modules.module.register_module_full_backward_pre_hook`	Register a backward pre-hook common to all the modules.
`tensorplay.nn.modules.module.register_module_full_backward_hook`	Register a backward hook common to all the modules.
`tensorplay.nn.modules.module.register_module_buffer_registration_hook`	Register a buffer registration hook common to all modules.
`tensorplay.nn.modules.module.register_module_module_registration_hook`	Register a module registration hook common to all modules.
`tensorplay.nn.modules.module.register_module_parameter_registration_hook`	Register a parameter registration hook common to all modules.

Convolution Layers

`tensorplay.nn.modules.conv.Conv1d`
`tensorplay.nn.modules.conv.Conv2d`
`tensorplay.nn.modules.conv.Conv3d`
`tensorplay.nn.modules.conv.ConvTranspose1d`
`tensorplay.nn.modules.conv.ConvTranspose2d`
`tensorplay.nn.modules.conv.ConvTranspose3d`
`tensorplay.nn.modules.conv.LazyConv1d`
`tensorplay.nn.modules.conv.LazyConv2d`
`tensorplay.nn.modules.conv.LazyConv3d`
`tensorplay.nn.modules.conv.LazyConvTranspose1d`
`tensorplay.nn.modules.conv.LazyConvTranspose2d`
`tensorplay.nn.modules.conv.LazyConvTranspose3d`
`tensorplay.nn.modules.folding.Unfold`	Extracts sliding local blocks from a batched input tensor (torch `torch.nn.Unfold`).
`tensorplay.nn.modules.folding.Fold`	Combines an array of sliding local blocks into a large containing tensor (torch `torch.nn.Fold`).

Pooling layers

`tensorplay.nn.modules.pooling.MaxPool1d`	Applies a 1D max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.MaxPool2d`	Applies a 2D max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.MaxPool3d`	Applies a 3D max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.MaxUnpool1d`	Computes a partial inverse of `MaxPool1d`.
`tensorplay.nn.modules.pooling.MaxUnpool2d`	Computes a partial inverse of `MaxPool2d`.
`tensorplay.nn.modules.pooling.MaxUnpool3d`	Computes a partial inverse of `MaxPool3d`.
`tensorplay.nn.modules.pooling.AvgPool1d`	Applies a 1D average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AvgPool2d`	Applies a 2D average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AvgPool3d`	Applies a 3D average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.FractionalMaxPool2d`	Applies a 2D fractional max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.FractionalMaxPool3d`	Applies a 3D fractional max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.LPPool1d`	Applies a 1D power-average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.LPPool2d`	Applies a 2D power-average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.LPPool3d`	Applies a 3D power-average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AdaptiveMaxPool1d`	Applies a 1D adaptive max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AdaptiveMaxPool2d`	Applies a 2D adaptive max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AdaptiveMaxPool3d`	Applies a 3D adaptive max pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AdaptiveAvgPool1d`	Applies a 1D adaptive average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AdaptiveAvgPool2d`	Applies a 2D adaptive average pooling over an input signal composed of several input planes.
`tensorplay.nn.modules.pooling.AdaptiveAvgPool3d`	Applies a 3D adaptive average pooling over an input signal composed of several input planes.

Padding Layers

`tensorplay.nn.modules.padding.ReflectionPad1d`	Pads the input tensor using the reflection of the input boundary.
`tensorplay.nn.modules.padding.ReflectionPad2d`	Pads the input tensor using the reflection of the input boundary.
`tensorplay.nn.modules.padding.ReflectionPad3d`	Pads the input tensor using the reflection of the input boundary.
`tensorplay.nn.modules.padding.ReplicationPad1d`	Pads the input tensor using replication of the input boundary.
`tensorplay.nn.modules.padding.ReplicationPad2d`	Pads the input tensor using replication of the input boundary.
`tensorplay.nn.modules.padding.ReplicationPad3d`	Pads the input tensor using replication of the input boundary.
`tensorplay.nn.modules.padding.ZeroPad1d`	Pads the input tensor boundaries with zero.
`tensorplay.nn.modules.padding.ZeroPad2d`	Pads the input tensor boundaries with zero.
`tensorplay.nn.modules.padding.ZeroPad3d`	Pads the input tensor boundaries with zero.
`tensorplay.nn.modules.padding.ConstantPad1d`	Pads the input tensor boundaries with a constant value.
`tensorplay.nn.modules.padding.ConstantPad2d`	Pads the input tensor boundaries with a constant value.
`tensorplay.nn.modules.padding.ConstantPad3d`	Pads the input tensor boundaries with a constant value.
`tensorplay.nn.modules.padding.CircularPad1d`	Pads the input tensor using circular padding of the input boundary.
`tensorplay.nn.modules.padding.CircularPad2d`	Pads the input tensor using circular padding of the input boundary.
`tensorplay.nn.modules.padding.CircularPad3d`	Pads the input tensor using circular padding of the input boundary.

Non-linear Activations (weighted sum, nonlinearity)

`tensorplay.nn.modules.activation.ELU`	Applies elu: `max(0, x) + min(0, alpha * (exp(x) - 1))`.
`tensorplay.nn.modules.activation.Hardshrink`	Applies the Hard Shrinkage (Hardshrink) function element-wise.
`tensorplay.nn.modules.activation.Hardsigmoid`	Applies hardsigmoid, element-wise: `ReLU6(x + 3) / 6`.
`tensorplay.nn.modules.activation.Hardtanh`	Applies the HardTanh function element-wise.
`tensorplay.nn.modules.activation.Hardswish`	Applies hardswish, element-wise: `x * ReLU6(x + 3) / 6`.
`tensorplay.nn.modules.activation.LeakyReLU`	Applies leaky_relu: `max(0, x) + negative_slope * min(0, x)`.
`tensorplay.nn.modules.activation.LogSigmoid`	Applies the Logsigmoid function element-wise.
`tensorplay.nn.modules.multihead_attention.MultiheadAttention`	Allows the model to jointly attend to information from different representation subspaces, as described in the paper Attention Is All You Need.
`tensorplay.nn.modules.activation.PReLU`	Applies the element-wise PReLU function.
`tensorplay.nn.modules.activation.ReLU`	Applies the rectified linear unit function element-wise.
`tensorplay.nn.modules.activation.ReLU6`	Applies the element-wise function `ReLU6(x) = min(max(0, x), 6)`.
`tensorplay.nn.modules.activation.RReLU`	Applies the randomized leaky rectified linear unit function, element-wise.
`tensorplay.nn.modules.activation.SELU`	Applies selu with ATen's fixed lambda/alpha constants.
`tensorplay.nn.modules.activation.CELU`	Applies celu: `max(0, x) + min(0, alpha * (exp(x / alpha) - 1))`.
`tensorplay.nn.modules.activation.Sigmoid`	Applies the Sigmoid function element-wise.
`tensorplay.nn.modules.activation.SiLU`	Applies the Sigmoid Linear Unit (SiLU) function, element-wise.
`tensorplay.nn.modules.activation.Mish`	Applies mish: `x * tanh(softplus(x))`.
`tensorplay.nn.modules.activation.Softplus`	Applies softplus with linearization above threshold * beta.
`tensorplay.nn.modules.activation.Softshrink`	Applies the soft shrinkage function element-wise.
`tensorplay.nn.modules.activation.Softsign`	Applies the element-wise function:
`tensorplay.nn.modules.activation.Tanh`	Applies the Hyperbolic Tangent (Tanh) function element-wise.
`tensorplay.nn.modules.activation.Tanhshrink`	Applies element-wise, \(\text{Tanhshrink}(x) = x - \text{Tanh}(x)\)
`tensorplay.nn.modules.activation.Threshold`	Thresholds each element of the input Tensor.
`tensorplay.nn.modules.activation.GLU`	Applies the Gaussian Error Linear Units function.

Non-linear Activations (other)

`tensorplay.nn.modules.activation.Softmin`	Applies the Softmin function to an n-dimensional input Tensor.
`tensorplay.nn.modules.activation.Softmax`	Softmax over `dim`, mirroring torch.nn.Softmax.
`tensorplay.nn.modules.activation.LogSoftmax`	Log-softmax over `dim`, mirroring torch.nn.LogSoftmax.
`tensorplay.nn.modules.adaptive.AdaptiveLogSoftmaxWithLoss`	Efficient softmax approximation.

Normalization Layers

`tensorplay.nn.modules.batchnorm.BatchNorm1d`	Applies Batch Normalization over a 2D or 3D input.
`tensorplay.nn.modules.batchnorm.BatchNorm2d`	Applies Batch Normalization over a 4D input.
`tensorplay.nn.modules.batchnorm.BatchNorm3d`	Applies Batch Normalization over a 5D input.
`tensorplay.nn.modules.batchnorm.LazyBatchNorm1d`	A `tensorplay.nn.BatchNorm1d` module with lazy initialization.
`tensorplay.nn.modules.batchnorm.LazyBatchNorm2d`	A `tensorplay.nn.BatchNorm2d` module with lazy initialization.
`tensorplay.nn.modules.batchnorm.LazyBatchNorm3d`	A `tensorplay.nn.BatchNorm3d` module with lazy initialization.
`tensorplay.nn.modules.normalization.GroupNorm`	Applies Group Normalization over a mini-batch of inputs.
`tensorplay.nn.modules.batchnorm.SyncBatchNorm`	Applies Batch Normalization over a N-Dimensional input with synchronized batch statistics across all processes in the group.
`tensorplay.nn.modules.instancenorm.InstanceNorm1d`	Applies Instance Normalization.
`tensorplay.nn.modules.instancenorm.InstanceNorm2d`	Applies Instance Normalization.
`tensorplay.nn.modules.instancenorm.InstanceNorm3d`	Applies Instance Normalization.
`tensorplay.nn.modules.instancenorm.LazyInstanceNorm1d`	A `tensorplay.nn.InstanceNorm1d` module with lazy initialization of the `num_features` argument.
`tensorplay.nn.modules.instancenorm.LazyInstanceNorm2d`	A `tensorplay.nn.InstanceNorm2d` module with lazy initialization of the `num_features` argument.
`tensorplay.nn.modules.instancenorm.LazyInstanceNorm3d`	A `tensorplay.nn.InstanceNorm3d` module with lazy initialization of the `num_features` argument.
`tensorplay.nn.modules.normalization.LayerNorm`	Applies Layer Normalization over a mini-batch of inputs.
`tensorplay.nn.modules.normalization.LocalResponseNorm`	Applies local response normalization over an input signal.
`tensorplay.nn.modules.normalization.RMSNorm`	Applies Root Mean Square Layer Normalization over a mini-batch of inputs.

Recurrent Layers

`tensorplay.nn.modules.rnn.RNNBase`	Base class for RNN modules (RNN, LSTM, GRU).
`tensorplay.nn.modules.rnn.RNN`	__init__(input_size, hidden_size, num_layers=1, nonlinearity='tanh', bias=True, batch_first=False, dropout=0.0, bidirectional=False, device=None, dtype=None)
`tensorplay.nn.modules.rnn.LSTM`	__init__(input_size, hidden_size, num_layers=1, bias=True, batch_first=False, dropout=0.0, bidirectional=False, proj_size=0, device=None, dtype=None)
`tensorplay.nn.modules.rnn.GRU`	__init__(input_size, hidden_size, num_layers=1, bias=True, batch_first=False, dropout=0.0, bidirectional=False, device=None, dtype=None)
`tensorplay.nn.modules.rnn.RNNCell`	An Elman RNN cell with tanh or ReLU non-linearity.
`tensorplay.nn.modules.rnn.LSTMCell`	A long short-term memory (LSTM) cell.
`tensorplay.nn.modules.rnn.GRUCell`	A gated recurrent unit (GRU) cell.

Transformer Layers

`tensorplay.nn.modules.transformer.Transformer`	A basic transformer layer.
`tensorplay.nn.modules.transformer.TransformerEncoder`	TransformerEncoder is a stack of N encoder layers.
`tensorplay.nn.modules.transformer.TransformerDecoder`	TransformerDecoder is a stack of N decoder layers.
`tensorplay.nn.modules.transformer.TransformerEncoderLayer`	TransformerEncoderLayer is made up of self-attn and feedforward network.
`tensorplay.nn.modules.transformer.TransformerDecoderLayer`	TransformerDecoderLayer is made up of self-attn, multi-head-attn and feedforward network.

Linear Layers

`tensorplay.nn.modules.linear.Identity`	A placeholder identity operator that is argument-insensitive.
`tensorplay.nn.modules.linear.Linear`	Applies an affine linear transformation to the incoming data: \(y = xA^T + b\).
`tensorplay.nn.modules.linear.Bilinear`	Applies a bilinear transformation to the incoming data: \(y = x_1^T A x_2 + b\).
`tensorplay.nn.modules.lazy.LazyLinear`	A `tensorplay.nn.Linear` module where in_features is inferred.

Dropout Layers

`tensorplay.nn.modules.dropout.Dropout`	During training, randomly zeroes some of the elements of the input tensor with probability `p`.
`tensorplay.nn.modules.dropout.Dropout1d`	Randomly zero out entire channels.
`tensorplay.nn.modules.dropout.Dropout2d`	Randomly zero out entire channels.
`tensorplay.nn.modules.dropout.Dropout3d`	Randomly zero out entire channels.
`tensorplay.nn.modules.dropout.AlphaDropout`	Applies Alpha Dropout over the input.
`tensorplay.nn.modules.dropout.FeatureAlphaDropout`	Randomly masks out entire channels.

Sparse Layers

`tensorplay.nn.modules.sparse.Embedding`
`tensorplay.nn.modules.sparse.EmbeddingBag`

Distance Functions

`tensorplay.nn.modules.distance.CosineSimilarity`	Returns cosine similarity between \(x_1\) and \(x_2\), computed along dim.
`tensorplay.nn.modules.distance.PairwiseDistance`	Computes the pairwise distance between input vectors, or between columns of input matrices.

Loss Functions

`tensorplay.nn.modules.loss.L1Loss`
`tensorplay.nn.modules.loss.MSELoss`
`tensorplay.nn.modules.loss.CrossEntropyLoss`
`tensorplay.nn.modules.loss.CTCLoss`
`tensorplay.nn.modules.loss.NLLLoss`
`tensorplay.nn.modules.loss.PoissonNLLLoss`
`tensorplay.nn.modules.loss.GaussianNLLLoss`
`tensorplay.nn.modules.loss.KLDivLoss`
`tensorplay.nn.modules.loss.BCELoss`
`tensorplay.nn.modules.loss.BCEWithLogitsLoss`
`tensorplay.nn.modules.loss.MarginRankingLoss`
`tensorplay.nn.modules.loss.HingeEmbeddingLoss`
`tensorplay.nn.modules.loss.MultiLabelMarginLoss`
`tensorplay.nn.modules.loss.HuberLoss`
`tensorplay.nn.modules.loss.SmoothL1Loss`
`tensorplay.nn.modules.loss.SoftMarginLoss`
`tensorplay.nn.modules.loss.MultiLabelSoftMarginLoss`
`tensorplay.nn.modules.loss.CosineEmbeddingLoss`
`tensorplay.nn.modules.loss.MultiMarginLoss`
`tensorplay.nn.modules.loss.TripletMarginLoss`
`tensorplay.nn.modules.loss.TripletMarginWithDistanceLoss`

Vision Layers

`tensorplay.nn.modules.pixelshuffle.PixelShuffle`	Rearrange elements in a tensor according to an upscaling factor.
`tensorplay.nn.modules.pixelshuffle.PixelUnshuffle`	Reverse the PixelShuffle operation.
`tensorplay.nn.modules.upsampling.Upsample`	Upsamples a given multi-channel 1D (temporal), 2D (spatial) or 3D (volumetric) data.
`tensorplay.nn.modules.upsampling.UpsamplingNearest2d`	Applies a 2D nearest neighbor upsampling to an input signal composed of several input channels.
`tensorplay.nn.modules.upsampling.UpsamplingBilinear2d`	Applies a 2D bilinear upsampling to an input signal composed of several input channels.

Shuffle Layers

tensorplay.nn.modules.channelshuffle.ChannelShuffle

Divides and rearranges the channels in a tensor.

DataParallel Layers (multi-GPU, distributed)

`tensorplay.nn.parallel.data_parallel.DataParallel`	Implements data parallelism at the module level.
`tensorplay.nn.parallel.distributed.DistributedDataParallel`	Implements distributed data parallelism (torch parity).

Utilities

`tensorplay.nn.utils.rnn.PackedSequence`	Holds the data and list of `batch_sizes` of a packed sequence.
`tensorplay.nn.utils.rnn.pack_padded_sequence`	Packs a Tensor containing padded sequences of variable length.
`tensorplay.nn.utils.rnn.pad_packed_sequence`	Pad a packed batch of variable length sequences.
`tensorplay.nn.utils.rnn.pad_sequence`	Pad a list of variable length Tensors with `padding_value`.
`tensorplay.nn.utils.rnn.pack_sequence`	Packs a list of variable length Tensors.
`tensorplay.nn.utils.rnn.unpack_sequence`	Unpack PackedSequence into a list of variable length Tensors.
`tensorplay.nn.utils.rnn.unpad_sequence`	Unpad padded Tensor into a list of variable length Tensors.
`tensorplay.nn.utils.rnn.invert_permutation`	Returns the inverse of `permutation`.
`tensorplay.nn.parameter.is_lazy`	Returns whether `param` is an `UninitializedParameter` or `UninitializedBuffer`.
`tensorplay.nn.factory_kwargs`	Return a canonicalized dict of factory kwargs.
`tensorplay.nn.modules.flatten.Flatten`	Flattens a contiguous range of dims into a tensor.
`tensorplay.nn.modules.flatten.Unflatten`	Unflattens a tensor dim expanding it to a desired shape.

Lazy Modules Initialization

tensorplay.nn.modules.lazy.LazyModuleMixin

A mixin for modules that lazily initialize parameters, also known as "lazy modules".

TensorPlay-specific additions

`Buffer`	A kind of Tensor that should not be considered a model parameter.
`DepthwiseConv2d`
`NonDynamicallyQuantizableLinear`
`Parameter`	A kind of Tensor that is to be considered a module parameter.
`RNNCellBase`
`UninitializedBuffer`	A buffer that is not initialized.
`UninitializedParameter`	A parameter that is not initialized.

tensorplay.nn

Containers

Convolution Layers

Pooling layers

Padding Layers

Non-linear Activations (weighted sum, nonlinearity)

Non-linear Activations (other)

Normalization Layers

Recurrent Layers

Transformer Layers

Linear Layers

Dropout Layers

Sparse Layers

Distance Functions

Loss Functions

Vision Layers

Shuffle Layers

DataParallel Layers (multi-GPU, distributed)

Utilities

Lazy Modules Initialization

TensorPlay-specific additions

Help improve this page