# tensorplay.nn

Source: https://www.tensorplay.cn/docs/nn.html

# tensorplay.nn

## Containers

[tensorplay.nn.modules.module.Module](generated/tensorplay.nn.modules.module.Module.html#tensorplay.nn.modules.module.Module)

Base class for all neural network modules.

[tensorplay.nn.modules.container.Sequential](generated/tensorplay.nn.modules.container.Sequential.html#tensorplay.nn.modules.container.Sequential)

A sequential container.

[tensorplay.nn.modules.container.ModuleList](generated/tensorplay.nn.modules.container.ModuleList.html#tensorplay.nn.modules.container.ModuleList)

Holds submodules in a list.

[tensorplay.nn.modules.container.ModuleDict](generated/tensorplay.nn.modules.container.ModuleDict.html#tensorplay.nn.modules.container.ModuleDict)

Holds submodules in a dictionary.

[tensorplay.nn.modules.container.ParameterList](generated/tensorplay.nn.modules.container.ParameterList.html#tensorplay.nn.modules.container.ParameterList)

Holds parameters in a list.

[tensorplay.nn.modules.container.ParameterDict](generated/tensorplay.nn.modules.container.ParameterDict.html#tensorplay.nn.modules.container.ParameterDict)

Holds parameters in a dictionary.

[tensorplay.nn.modules.module.register_module_forward_pre_hook](generated/tensorplay.nn.modules.module.register_module_forward_pre_hook.html#tensorplay.nn.modules.module.register_module_forward_pre_hook)

Register a forward pre-hook common to all modules.

[tensorplay.nn.modules.module.register_module_forward_hook](generated/tensorplay.nn.modules.module.register_module_forward_hook.html#tensorplay.nn.modules.module.register_module_forward_hook)

Register a global forward hook for all the modules.

[tensorplay.nn.modules.module.register_module_backward_hook](generated/tensorplay.nn.modules.module.register_module_backward_hook.html#tensorplay.nn.modules.module.register_module_backward_hook)

Register a backward hook common to all the modules.

[tensorplay.nn.modules.module.register_module_full_backward_pre_hook](generated/tensorplay.nn.modules.module.register_module_full_backward_pre_hook.html#tensorplay.nn.modules.module.register_module_full_backward_pre_hook)

Register a backward pre-hook common to all the modules.

[tensorplay.nn.modules.module.register_module_full_backward_hook](generated/tensorplay.nn.modules.module.register_module_full_backward_hook.html#tensorplay.nn.modules.module.register_module_full_backward_hook)

Register a backward hook common to all the modules.

[tensorplay.nn.modules.module.register_module_buffer_registration_hook](generated/tensorplay.nn.modules.module.register_module_buffer_registration_hook.html#tensorplay.nn.modules.module.register_module_buffer_registration_hook)

Register a buffer registration hook common to all modules.

[tensorplay.nn.modules.module.register_module_module_registration_hook](generated/tensorplay.nn.modules.module.register_module_module_registration_hook.html#tensorplay.nn.modules.module.register_module_module_registration_hook)

Register a module registration hook common to all modules.

[tensorplay.nn.modules.module.register_module_parameter_registration_hook](generated/tensorplay.nn.modules.module.register_module_parameter_registration_hook.html#tensorplay.nn.modules.module.register_module_parameter_registration_hook)

Register a parameter registration hook common to all modules.

## Convolution Layers

[tensorplay.nn.modules.conv.Conv1d](generated/tensorplay.nn.modules.conv.Conv1d.html#tensorplay.nn.modules.conv.Conv1d)

[tensorplay.nn.modules.conv.Conv2d](generated/tensorplay.nn.modules.conv.Conv2d.html#tensorplay.nn.modules.conv.Conv2d)

[tensorplay.nn.modules.conv.Conv3d](generated/tensorplay.nn.modules.conv.Conv3d.html#tensorplay.nn.modules.conv.Conv3d)

[tensorplay.nn.modules.conv.ConvTranspose1d](generated/tensorplay.nn.modules.conv.ConvTranspose1d.html#tensorplay.nn.modules.conv.ConvTranspose1d)

[tensorplay.nn.modules.conv.ConvTranspose2d](generated/tensorplay.nn.modules.conv.ConvTranspose2d.html#tensorplay.nn.modules.conv.ConvTranspose2d)

[tensorplay.nn.modules.conv.ConvTranspose3d](generated/tensorplay.nn.modules.conv.ConvTranspose3d.html#tensorplay.nn.modules.conv.ConvTranspose3d)

[tensorplay.nn.modules.conv.LazyConv1d](generated/tensorplay.nn.modules.conv.LazyConv1d.html#tensorplay.nn.modules.conv.LazyConv1d)

[tensorplay.nn.modules.conv.LazyConv2d](generated/tensorplay.nn.modules.conv.LazyConv2d.html#tensorplay.nn.modules.conv.LazyConv2d)

[tensorplay.nn.modules.conv.LazyConv3d](generated/tensorplay.nn.modules.conv.LazyConv3d.html#tensorplay.nn.modules.conv.LazyConv3d)

[tensorplay.nn.modules.conv.LazyConvTranspose1d](generated/tensorplay.nn.modules.conv.LazyConvTranspose1d.html#tensorplay.nn.modules.conv.LazyConvTranspose1d)

[tensorplay.nn.modules.conv.LazyConvTranspose2d](generated/tensorplay.nn.modules.conv.LazyConvTranspose2d.html#tensorplay.nn.modules.conv.LazyConvTranspose2d)

[tensorplay.nn.modules.conv.LazyConvTranspose3d](generated/tensorplay.nn.modules.conv.LazyConvTranspose3d.html#tensorplay.nn.modules.conv.LazyConvTranspose3d)

[tensorplay.nn.modules.folding.Unfold](generated/tensorplay.nn.modules.folding.Unfold.html#tensorplay.nn.modules.folding.Unfold)

Extracts sliding local blocks from a batched input tensor (torch torch.nn.Unfold).

[tensorplay.nn.modules.folding.Fold](generated/tensorplay.nn.modules.folding.Fold.html#tensorplay.nn.modules.folding.Fold)

Combines an array of sliding local blocks into a large containing tensor (torch torch.nn.Fold).

## Pooling layers

[tensorplay.nn.modules.pooling.MaxPool1d](generated/tensorplay.nn.modules.pooling.MaxPool1d.html#tensorplay.nn.modules.pooling.MaxPool1d)

Applies a 1D max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.MaxPool2d](generated/tensorplay.nn.modules.pooling.MaxPool2d.html#tensorplay.nn.modules.pooling.MaxPool2d)

Applies a 2D max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.MaxPool3d](generated/tensorplay.nn.modules.pooling.MaxPool3d.html#tensorplay.nn.modules.pooling.MaxPool3d)

Applies a 3D max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.MaxUnpool1d](generated/tensorplay.nn.modules.pooling.MaxUnpool1d.html#tensorplay.nn.modules.pooling.MaxUnpool1d)

Computes a partial inverse of MaxPool1d.

[tensorplay.nn.modules.pooling.MaxUnpool2d](generated/tensorplay.nn.modules.pooling.MaxUnpool2d.html#tensorplay.nn.modules.pooling.MaxUnpool2d)

Computes a partial inverse of MaxPool2d.

[tensorplay.nn.modules.pooling.MaxUnpool3d](generated/tensorplay.nn.modules.pooling.MaxUnpool3d.html#tensorplay.nn.modules.pooling.MaxUnpool3d)

Computes a partial inverse of MaxPool3d.

[tensorplay.nn.modules.pooling.AvgPool1d](generated/tensorplay.nn.modules.pooling.AvgPool1d.html#tensorplay.nn.modules.pooling.AvgPool1d)

Applies a 1D average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AvgPool2d](generated/tensorplay.nn.modules.pooling.AvgPool2d.html#tensorplay.nn.modules.pooling.AvgPool2d)

Applies a 2D average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AvgPool3d](generated/tensorplay.nn.modules.pooling.AvgPool3d.html#tensorplay.nn.modules.pooling.AvgPool3d)

Applies a 3D average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.FractionalMaxPool2d](generated/tensorplay.nn.modules.pooling.FractionalMaxPool2d.html#tensorplay.nn.modules.pooling.FractionalMaxPool2d)

Applies a 2D fractional max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.FractionalMaxPool3d](generated/tensorplay.nn.modules.pooling.FractionalMaxPool3d.html#tensorplay.nn.modules.pooling.FractionalMaxPool3d)

Applies a 3D fractional max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.LPPool1d](generated/tensorplay.nn.modules.pooling.LPPool1d.html#tensorplay.nn.modules.pooling.LPPool1d)

Applies a 1D power-average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.LPPool2d](generated/tensorplay.nn.modules.pooling.LPPool2d.html#tensorplay.nn.modules.pooling.LPPool2d)

Applies a 2D power-average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.LPPool3d](generated/tensorplay.nn.modules.pooling.LPPool3d.html#tensorplay.nn.modules.pooling.LPPool3d)

Applies a 3D power-average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AdaptiveMaxPool1d](generated/tensorplay.nn.modules.pooling.AdaptiveMaxPool1d.html#tensorplay.nn.modules.pooling.AdaptiveMaxPool1d)

Applies a 1D adaptive max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AdaptiveMaxPool2d](generated/tensorplay.nn.modules.pooling.AdaptiveMaxPool2d.html#tensorplay.nn.modules.pooling.AdaptiveMaxPool2d)

Applies a 2D adaptive max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AdaptiveMaxPool3d](generated/tensorplay.nn.modules.pooling.AdaptiveMaxPool3d.html#tensorplay.nn.modules.pooling.AdaptiveMaxPool3d)

Applies a 3D adaptive max pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AdaptiveAvgPool1d](generated/tensorplay.nn.modules.pooling.AdaptiveAvgPool1d.html#tensorplay.nn.modules.pooling.AdaptiveAvgPool1d)

Applies a 1D adaptive average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AdaptiveAvgPool2d](generated/tensorplay.nn.modules.pooling.AdaptiveAvgPool2d.html#tensorplay.nn.modules.pooling.AdaptiveAvgPool2d)

Applies a 2D adaptive average pooling over an input signal composed of several input planes.

[tensorplay.nn.modules.pooling.AdaptiveAvgPool3d](generated/tensorplay.nn.modules.pooling.AdaptiveAvgPool3d.html#tensorplay.nn.modules.pooling.AdaptiveAvgPool3d)

Applies a 3D adaptive average pooling over an input signal composed of several input planes.

## Padding Layers

[tensorplay.nn.modules.padding.ReflectionPad1d](generated/tensorplay.nn.modules.padding.ReflectionPad1d.html#tensorplay.nn.modules.padding.ReflectionPad1d)

Pads the input tensor using the reflection of the input boundary.

[tensorplay.nn.modules.padding.ReflectionPad2d](generated/tensorplay.nn.modules.padding.ReflectionPad2d.html#tensorplay.nn.modules.padding.ReflectionPad2d)

Pads the input tensor using the reflection of the input boundary.

[tensorplay.nn.modules.padding.ReflectionPad3d](generated/tensorplay.nn.modules.padding.ReflectionPad3d.html#tensorplay.nn.modules.padding.ReflectionPad3d)

Pads the input tensor using the reflection of the input boundary.

[tensorplay.nn.modules.padding.ReplicationPad1d](generated/tensorplay.nn.modules.padding.ReplicationPad1d.html#tensorplay.nn.modules.padding.ReplicationPad1d)

Pads the input tensor using replication of the input boundary.

[tensorplay.nn.modules.padding.ReplicationPad2d](generated/tensorplay.nn.modules.padding.ReplicationPad2d.html#tensorplay.nn.modules.padding.ReplicationPad2d)

Pads the input tensor using replication of the input boundary.

[tensorplay.nn.modules.padding.ReplicationPad3d](generated/tensorplay.nn.modules.padding.ReplicationPad3d.html#tensorplay.nn.modules.padding.ReplicationPad3d)

Pads the input tensor using replication of the input boundary.

[tensorplay.nn.modules.padding.ZeroPad1d](generated/tensorplay.nn.modules.padding.ZeroPad1d.html#tensorplay.nn.modules.padding.ZeroPad1d)

Pads the input tensor boundaries with zero.

[tensorplay.nn.modules.padding.ZeroPad2d](generated/tensorplay.nn.modules.padding.ZeroPad2d.html#tensorplay.nn.modules.padding.ZeroPad2d)

Pads the input tensor boundaries with zero.

[tensorplay.nn.modules.padding.ZeroPad3d](generated/tensorplay.nn.modules.padding.ZeroPad3d.html#tensorplay.nn.modules.padding.ZeroPad3d)

Pads the input tensor boundaries with zero.

[tensorplay.nn.modules.padding.ConstantPad1d](generated/tensorplay.nn.modules.padding.ConstantPad1d.html#tensorplay.nn.modules.padding.ConstantPad1d)

Pads the input tensor boundaries with a constant value.

[tensorplay.nn.modules.padding.ConstantPad2d](generated/tensorplay.nn.modules.padding.ConstantPad2d.html#tensorplay.nn.modules.padding.ConstantPad2d)

Pads the input tensor boundaries with a constant value.

[tensorplay.nn.modules.padding.ConstantPad3d](generated/tensorplay.nn.modules.padding.ConstantPad3d.html#tensorplay.nn.modules.padding.ConstantPad3d)

Pads the input tensor boundaries with a constant value.

[tensorplay.nn.modules.padding.CircularPad1d](generated/tensorplay.nn.modules.padding.CircularPad1d.html#tensorplay.nn.modules.padding.CircularPad1d)

Pads the input tensor using circular padding of the input boundary.

[tensorplay.nn.modules.padding.CircularPad2d](generated/tensorplay.nn.modules.padding.CircularPad2d.html#tensorplay.nn.modules.padding.CircularPad2d)

Pads the input tensor using circular padding of the input boundary.

[tensorplay.nn.modules.padding.CircularPad3d](generated/tensorplay.nn.modules.padding.CircularPad3d.html#tensorplay.nn.modules.padding.CircularPad3d)

Pads the input tensor using circular padding of the input boundary.

## Non-linear Activations (weighted sum, nonlinearity)

[tensorplay.nn.modules.activation.ELU](generated/tensorplay.nn.modules.activation.ELU.html#tensorplay.nn.modules.activation.ELU)

Applies elu: max(0, x) + min(0, alpha * (exp(x) - 1)).

[tensorplay.nn.modules.activation.Hardshrink](generated/tensorplay.nn.modules.activation.Hardshrink.html#tensorplay.nn.modules.activation.Hardshrink)

Applies the Hard Shrinkage (Hardshrink) function element-wise.

[tensorplay.nn.modules.activation.Hardsigmoid](generated/tensorplay.nn.modules.activation.Hardsigmoid.html#tensorplay.nn.modules.activation.Hardsigmoid)

Applies hardsigmoid, element-wise: ReLU6(x + 3) / 6.

[tensorplay.nn.modules.activation.Hardtanh](generated/tensorplay.nn.modules.activation.Hardtanh.html#tensorplay.nn.modules.activation.Hardtanh)

Applies the HardTanh function element-wise.

[tensorplay.nn.modules.activation.Hardswish](generated/tensorplay.nn.modules.activation.Hardswish.html#tensorplay.nn.modules.activation.Hardswish)

Applies hardswish, element-wise: x * ReLU6(x + 3) / 6.

[tensorplay.nn.modules.activation.LeakyReLU](generated/tensorplay.nn.modules.activation.LeakyReLU.html#tensorplay.nn.modules.activation.LeakyReLU)

Applies leaky_relu: max(0, x) + negative_slope * min(0, x).

[tensorplay.nn.modules.activation.LogSigmoid](generated/tensorplay.nn.modules.activation.LogSigmoid.html#tensorplay.nn.modules.activation.LogSigmoid)

Applies the Logsigmoid function element-wise.

[tensorplay.nn.modules.multihead_attention.MultiheadAttention](generated/tensorplay.nn.modules.multihead_attention.MultiheadAttention.html#tensorplay.nn.modules.multihead_attention.MultiheadAttention)

Allows the model to jointly attend to information from different representation subspaces, as described in the paper Attention Is All You Need.

[tensorplay.nn.modules.activation.PReLU](generated/tensorplay.nn.modules.activation.PReLU.html#tensorplay.nn.modules.activation.PReLU)

Applies the element-wise PReLU function.

[tensorplay.nn.modules.activation.ReLU](generated/tensorplay.nn.modules.activation.ReLU.html#tensorplay.nn.modules.activation.ReLU)

Applies the rectified linear unit function element-wise.

[tensorplay.nn.modules.activation.ReLU6](generated/tensorplay.nn.modules.activation.ReLU6.html#tensorplay.nn.modules.activation.ReLU6)

Applies the element-wise function ReLU6(x) = min(max(0, x), 6).

[tensorplay.nn.modules.activation.RReLU](generated/tensorplay.nn.modules.activation.RReLU.html#tensorplay.nn.modules.activation.RReLU)

Applies the randomized leaky rectified linear unit function, element-wise.

[tensorplay.nn.modules.activation.SELU](generated/tensorplay.nn.modules.activation.SELU.html#tensorplay.nn.modules.activation.SELU)

Applies selu with ATen's fixed lambda/alpha constants.

[tensorplay.nn.modules.activation.CELU](generated/tensorplay.nn.modules.activation.CELU.html#tensorplay.nn.modules.activation.CELU)

Applies celu: max(0, x) + min(0, alpha * (exp(x / alpha) - 1)).

[tensorplay.nn.modules.activation.Sigmoid](generated/tensorplay.nn.modules.activation.Sigmoid.html#tensorplay.nn.modules.activation.Sigmoid)

Applies the Sigmoid function element-wise.

[tensorplay.nn.modules.activation.SiLU](generated/tensorplay.nn.modules.activation.SiLU.html#tensorplay.nn.modules.activation.SiLU)

Applies the Sigmoid Linear Unit (SiLU) function, element-wise.

[tensorplay.nn.modules.activation.Mish](generated/tensorplay.nn.modules.activation.Mish.html#tensorplay.nn.modules.activation.Mish)

Applies mish: x * tanh(softplus(x)).

[tensorplay.nn.modules.activation.Softplus](generated/tensorplay.nn.modules.activation.Softplus.html#tensorplay.nn.modules.activation.Softplus)

Applies softplus with linearization above threshold * beta.

[tensorplay.nn.modules.activation.Softshrink](generated/tensorplay.nn.modules.activation.Softshrink.html#tensorplay.nn.modules.activation.Softshrink)

Applies the soft shrinkage function element-wise.

[tensorplay.nn.modules.activation.Softsign](generated/tensorplay.nn.modules.activation.Softsign.html#tensorplay.nn.modules.activation.Softsign)

Applies the element-wise function:

[tensorplay.nn.modules.activation.Tanh](generated/tensorplay.nn.modules.activation.Tanh.html#tensorplay.nn.modules.activation.Tanh)

Applies the Hyperbolic Tangent (Tanh) function element-wise.

[tensorplay.nn.modules.activation.Tanhshrink](generated/tensorplay.nn.modules.activation.Tanhshrink.html#tensorplay.nn.modules.activation.Tanhshrink)

Applies element-wise, \(\text{Tanhshrink}(x) = x - \text{Tanh}(x)\)

[tensorplay.nn.modules.activation.Threshold](generated/tensorplay.nn.modules.activation.Threshold.html#tensorplay.nn.modules.activation.Threshold)

Thresholds each element of the input Tensor.

[tensorplay.nn.modules.activation.GLU](generated/tensorplay.nn.modules.activation.GLU.html#tensorplay.nn.modules.activation.GLU)

Applies the Gaussian Error Linear Units function.

## Non-linear Activations (other)

[tensorplay.nn.modules.activation.Softmin](generated/tensorplay.nn.modules.activation.Softmin.html#tensorplay.nn.modules.activation.Softmin)

Applies the Softmin function to an n-dimensional input Tensor.

[tensorplay.nn.modules.activation.Softmax](generated/tensorplay.nn.modules.activation.Softmax.html#tensorplay.nn.modules.activation.Softmax)

Softmax over dim, mirroring torch.nn.Softmax.

[tensorplay.nn.modules.activation.LogSoftmax](generated/tensorplay.nn.modules.activation.LogSoftmax.html#tensorplay.nn.modules.activation.LogSoftmax)

Log-softmax over dim, mirroring torch.nn.LogSoftmax.

[tensorplay.nn.modules.adaptive.AdaptiveLogSoftmaxWithLoss](generated/tensorplay.nn.modules.adaptive.AdaptiveLogSoftmaxWithLoss.html#tensorplay.nn.modules.adaptive.AdaptiveLogSoftmaxWithLoss)

Efficient softmax approximation.

## Normalization Layers

[tensorplay.nn.modules.batchnorm.BatchNorm1d](generated/tensorplay.nn.modules.batchnorm.BatchNorm1d.html#tensorplay.nn.modules.batchnorm.BatchNorm1d)

Applies Batch Normalization over a 2D or 3D input.

[tensorplay.nn.modules.batchnorm.BatchNorm2d](generated/tensorplay.nn.modules.batchnorm.BatchNorm2d.html#tensorplay.nn.modules.batchnorm.BatchNorm2d)

Applies Batch Normalization over a 4D input.

[tensorplay.nn.modules.batchnorm.BatchNorm3d](generated/tensorplay.nn.modules.batchnorm.BatchNorm3d.html#tensorplay.nn.modules.batchnorm.BatchNorm3d)

Applies Batch Normalization over a 5D input.

[tensorplay.nn.modules.batchnorm.LazyBatchNorm1d](generated/tensorplay.nn.modules.batchnorm.LazyBatchNorm1d.html#tensorplay.nn.modules.batchnorm.LazyBatchNorm1d)

A tensorplay.nn.BatchNorm1d module with lazy initialization.

[tensorplay.nn.modules.batchnorm.LazyBatchNorm2d](generated/tensorplay.nn.modules.batchnorm.LazyBatchNorm2d.html#tensorplay.nn.modules.batchnorm.LazyBatchNorm2d)

A tensorplay.nn.BatchNorm2d module with lazy initialization.

[tensorplay.nn.modules.batchnorm.LazyBatchNorm3d](generated/tensorplay.nn.modules.batchnorm.LazyBatchNorm3d.html#tensorplay.nn.modules.batchnorm.LazyBatchNorm3d)

A tensorplay.nn.BatchNorm3d module with lazy initialization.

[tensorplay.nn.modules.normalization.GroupNorm](generated/tensorplay.nn.modules.normalization.GroupNorm.html#tensorplay.nn.modules.normalization.GroupNorm)

Applies Group Normalization over a mini-batch of inputs.

[tensorplay.nn.modules.batchnorm.SyncBatchNorm](generated/tensorplay.nn.modules.batchnorm.SyncBatchNorm.html#tensorplay.nn.modules.batchnorm.SyncBatchNorm)

Applies Batch Normalization over a N-Dimensional input with synchronized batch statistics across all processes in the group.

[tensorplay.nn.modules.instancenorm.InstanceNorm1d](generated/tensorplay.nn.modules.instancenorm.InstanceNorm1d.html#tensorplay.nn.modules.instancenorm.InstanceNorm1d)

Applies Instance Normalization.

[tensorplay.nn.modules.instancenorm.InstanceNorm2d](generated/tensorplay.nn.modules.instancenorm.InstanceNorm2d.html#tensorplay.nn.modules.instancenorm.InstanceNorm2d)

Applies Instance Normalization.

[tensorplay.nn.modules.instancenorm.InstanceNorm3d](generated/tensorplay.nn.modules.instancenorm.InstanceNorm3d.html#tensorplay.nn.modules.instancenorm.InstanceNorm3d)

Applies Instance Normalization.

[tensorplay.nn.modules.instancenorm.LazyInstanceNorm1d](generated/tensorplay.nn.modules.instancenorm.LazyInstanceNorm1d.html#tensorplay.nn.modules.instancenorm.LazyInstanceNorm1d)

A tensorplay.nn.InstanceNorm1d module with lazy initialization of the num_features argument.

[tensorplay.nn.modules.instancenorm.LazyInstanceNorm2d](generated/tensorplay.nn.modules.instancenorm.LazyInstanceNorm2d.html#tensorplay.nn.modules.instancenorm.LazyInstanceNorm2d)

A tensorplay.nn.InstanceNorm2d module with lazy initialization of the num_features argument.

[tensorplay.nn.modules.instancenorm.LazyInstanceNorm3d](generated/tensorplay.nn.modules.instancenorm.LazyInstanceNorm3d.html#tensorplay.nn.modules.instancenorm.LazyInstanceNorm3d)

A tensorplay.nn.InstanceNorm3d module with lazy initialization of the num_features argument.

[tensorplay.nn.modules.normalization.LayerNorm](generated/tensorplay.nn.modules.normalization.LayerNorm.html#tensorplay.nn.modules.normalization.LayerNorm)

Applies Layer Normalization over a mini-batch of inputs.

[tensorplay.nn.modules.normalization.LocalResponseNorm](generated/tensorplay.nn.modules.normalization.LocalResponseNorm.html#tensorplay.nn.modules.normalization.LocalResponseNorm)

Applies local response normalization over an input signal.

[tensorplay.nn.modules.normalization.RMSNorm](generated/tensorplay.nn.modules.normalization.RMSNorm.html#tensorplay.nn.modules.normalization.RMSNorm)

Applies Root Mean Square Layer Normalization over a mini-batch of inputs.

## Recurrent Layers

[tensorplay.nn.modules.rnn.RNNBase](generated/tensorplay.nn.modules.rnn.RNNBase.html#tensorplay.nn.modules.rnn.RNNBase)

Base class for RNN modules (RNN, LSTM, GRU).

[tensorplay.nn.modules.rnn.RNN](generated/tensorplay.nn.modules.rnn.RNN.html#tensorplay.nn.modules.rnn.RNN)

__init__(input_size, hidden_size, num_layers=1, nonlinearity='tanh', bias=True, batch_first=False, dropout=0.0, bidirectional=False, device=None, dtype=None)

[tensorplay.nn.modules.rnn.LSTM](generated/tensorplay.nn.modules.rnn.LSTM.html#tensorplay.nn.modules.rnn.LSTM)

__init__(input_size, hidden_size, num_layers=1, bias=True, batch_first=False, dropout=0.0, bidirectional=False, proj_size=0, device=None, dtype=None)

[tensorplay.nn.modules.rnn.GRU](generated/tensorplay.nn.modules.rnn.GRU.html#tensorplay.nn.modules.rnn.GRU)

__init__(input_size, hidden_size, num_layers=1, bias=True, batch_first=False, dropout=0.0, bidirectional=False, device=None, dtype=None)

[tensorplay.nn.modules.rnn.RNNCell](generated/tensorplay.nn.modules.rnn.RNNCell.html#tensorplay.nn.modules.rnn.RNNCell)

An Elman RNN cell with tanh or ReLU non-linearity.

[tensorplay.nn.modules.rnn.LSTMCell](generated/tensorplay.nn.modules.rnn.LSTMCell.html#tensorplay.nn.modules.rnn.LSTMCell)

A long short-term memory (LSTM) cell.

[tensorplay.nn.modules.rnn.GRUCell](generated/tensorplay.nn.modules.rnn.GRUCell.html#tensorplay.nn.modules.rnn.GRUCell)

A gated recurrent unit (GRU) cell.

## Transformer Layers

[tensorplay.nn.modules.transformer.Transformer](generated/tensorplay.nn.modules.transformer.Transformer.html#tensorplay.nn.modules.transformer.Transformer)

A basic transformer layer.

[tensorplay.nn.modules.transformer.TransformerEncoder](generated/tensorplay.nn.modules.transformer.TransformerEncoder.html#tensorplay.nn.modules.transformer.TransformerEncoder)

TransformerEncoder is a stack of N encoder layers.

[tensorplay.nn.modules.transformer.TransformerDecoder](generated/tensorplay.nn.modules.transformer.TransformerDecoder.html#tensorplay.nn.modules.transformer.TransformerDecoder)

TransformerDecoder is a stack of N decoder layers.

[tensorplay.nn.modules.transformer.TransformerEncoderLayer](generated/tensorplay.nn.modules.transformer.TransformerEncoderLayer.html#tensorplay.nn.modules.transformer.TransformerEncoderLayer)

TransformerEncoderLayer is made up of self-attn and feedforward network.

[tensorplay.nn.modules.transformer.TransformerDecoderLayer](generated/tensorplay.nn.modules.transformer.TransformerDecoderLayer.html#tensorplay.nn.modules.transformer.TransformerDecoderLayer)

TransformerDecoderLayer is made up of self-attn, multi-head-attn and feedforward network.

## Linear Layers

[tensorplay.nn.modules.linear.Identity](generated/tensorplay.nn.modules.linear.Identity.html#tensorplay.nn.modules.linear.Identity)

A placeholder identity operator that is argument-insensitive.

[tensorplay.nn.modules.linear.Linear](generated/tensorplay.nn.modules.linear.Linear.html#tensorplay.nn.modules.linear.Linear)

Applies an affine linear transformation to the incoming data: \(y = xA^T + b\).

[tensorplay.nn.modules.linear.Bilinear](generated/tensorplay.nn.modules.linear.Bilinear.html#tensorplay.nn.modules.linear.Bilinear)

Applies a bilinear transformation to the incoming data: \(y = x_1^T A x_2 + b\).

[tensorplay.nn.modules.lazy.LazyLinear](generated/tensorplay.nn.modules.lazy.LazyLinear.html#tensorplay.nn.modules.lazy.LazyLinear)

A tensorplay.nn.Linear module where in_features is inferred.

## Dropout Layers

[tensorplay.nn.modules.dropout.Dropout](generated/tensorplay.nn.modules.dropout.Dropout.html#tensorplay.nn.modules.dropout.Dropout)

During training, randomly zeroes some of the elements of the input tensor with probability p.

[tensorplay.nn.modules.dropout.Dropout1d](generated/tensorplay.nn.modules.dropout.Dropout1d.html#tensorplay.nn.modules.dropout.Dropout1d)

Randomly zero out entire channels.

[tensorplay.nn.modules.dropout.Dropout2d](generated/tensorplay.nn.modules.dropout.Dropout2d.html#tensorplay.nn.modules.dropout.Dropout2d)

Randomly zero out entire channels.

[tensorplay.nn.modules.dropout.Dropout3d](generated/tensorplay.nn.modules.dropout.Dropout3d.html#tensorplay.nn.modules.dropout.Dropout3d)

Randomly zero out entire channels.

[tensorplay.nn.modules.dropout.AlphaDropout](generated/tensorplay.nn.modules.dropout.AlphaDropout.html#tensorplay.nn.modules.dropout.AlphaDropout)

Applies Alpha Dropout over the input.

[tensorplay.nn.modules.dropout.FeatureAlphaDropout](generated/tensorplay.nn.modules.dropout.FeatureAlphaDropout.html#tensorplay.nn.modules.dropout.FeatureAlphaDropout)

Randomly masks out entire channels.

## Sparse Layers

[tensorplay.nn.modules.sparse.Embedding](generated/tensorplay.nn.modules.sparse.Embedding.html#tensorplay.nn.modules.sparse.Embedding)

[tensorplay.nn.modules.sparse.EmbeddingBag](generated/tensorplay.nn.modules.sparse.EmbeddingBag.html#tensorplay.nn.modules.sparse.EmbeddingBag)

## Distance Functions

[tensorplay.nn.modules.distance.CosineSimilarity](generated/tensorplay.nn.modules.distance.CosineSimilarity.html#tensorplay.nn.modules.distance.CosineSimilarity)

Returns cosine similarity between \(x_1\) and \(x_2\), computed along dim.

[tensorplay.nn.modules.distance.PairwiseDistance](generated/tensorplay.nn.modules.distance.PairwiseDistance.html#tensorplay.nn.modules.distance.PairwiseDistance)

Computes the pairwise distance between input vectors, or between columns of input matrices.

## Loss Functions

[tensorplay.nn.modules.loss.L1Loss](generated/tensorplay.nn.modules.loss.L1Loss.html#tensorplay.nn.modules.loss.L1Loss)

[tensorplay.nn.modules.loss.MSELoss](generated/tensorplay.nn.modules.loss.MSELoss.html#tensorplay.nn.modules.loss.MSELoss)

[tensorplay.nn.modules.loss.CrossEntropyLoss](generated/tensorplay.nn.modules.loss.CrossEntropyLoss.html#tensorplay.nn.modules.loss.CrossEntropyLoss)

[tensorplay.nn.modules.loss.CTCLoss](generated/tensorplay.nn.modules.loss.CTCLoss.html#tensorplay.nn.modules.loss.CTCLoss)

[tensorplay.nn.modules.loss.NLLLoss](generated/tensorplay.nn.modules.loss.NLLLoss.html#tensorplay.nn.modules.loss.NLLLoss)

[tensorplay.nn.modules.loss.PoissonNLLLoss](generated/tensorplay.nn.modules.loss.PoissonNLLLoss.html#tensorplay.nn.modules.loss.PoissonNLLLoss)

[tensorplay.nn.modules.loss.GaussianNLLLoss](generated/tensorplay.nn.modules.loss.GaussianNLLLoss.html#tensorplay.nn.modules.loss.GaussianNLLLoss)

[tensorplay.nn.modules.loss.KLDivLoss](generated/tensorplay.nn.modules.loss.KLDivLoss.html#tensorplay.nn.modules.loss.KLDivLoss)

[tensorplay.nn.modules.loss.BCELoss](generated/tensorplay.nn.modules.loss.BCELoss.html#tensorplay.nn.modules.loss.BCELoss)

[tensorplay.nn.modules.loss.BCEWithLogitsLoss](generated/tensorplay.nn.modules.loss.BCEWithLogitsLoss.html#tensorplay.nn.modules.loss.BCEWithLogitsLoss)

[tensorplay.nn.modules.loss.MarginRankingLoss](generated/tensorplay.nn.modules.loss.MarginRankingLoss.html#tensorplay.nn.modules.loss.MarginRankingLoss)

[tensorplay.nn.modules.loss.HingeEmbeddingLoss](generated/tensorplay.nn.modules.loss.HingeEmbeddingLoss.html#tensorplay.nn.modules.loss.HingeEmbeddingLoss)

[tensorplay.nn.modules.loss.MultiLabelMarginLoss](generated/tensorplay.nn.modules.loss.MultiLabelMarginLoss.html#tensorplay.nn.modules.loss.MultiLabelMarginLoss)

[tensorplay.nn.modules.loss.HuberLoss](generated/tensorplay.nn.modules.loss.HuberLoss.html#tensorplay.nn.modules.loss.HuberLoss)

[tensorplay.nn.modules.loss.SmoothL1Loss](generated/tensorplay.nn.modules.loss.SmoothL1Loss.html#tensorplay.nn.modules.loss.SmoothL1Loss)

[tensorplay.nn.modules.loss.SoftMarginLoss](generated/tensorplay.nn.modules.loss.SoftMarginLoss.html#tensorplay.nn.modules.loss.SoftMarginLoss)

[tensorplay.nn.modules.loss.MultiLabelSoftMarginLoss](generated/tensorplay.nn.modules.loss.MultiLabelSoftMarginLoss.html#tensorplay.nn.modules.loss.MultiLabelSoftMarginLoss)

[tensorplay.nn.modules.loss.CosineEmbeddingLoss](generated/tensorplay.nn.modules.loss.CosineEmbeddingLoss.html#tensorplay.nn.modules.loss.CosineEmbeddingLoss)

[tensorplay.nn.modules.loss.MultiMarginLoss](generated/tensorplay.nn.modules.loss.MultiMarginLoss.html#tensorplay.nn.modules.loss.MultiMarginLoss)

[tensorplay.nn.modules.loss.TripletMarginLoss](generated/tensorplay.nn.modules.loss.TripletMarginLoss.html#tensorplay.nn.modules.loss.TripletMarginLoss)

[tensorplay.nn.modules.loss.TripletMarginWithDistanceLoss](generated/tensorplay.nn.modules.loss.TripletMarginWithDistanceLoss.html#tensorplay.nn.modules.loss.TripletMarginWithDistanceLoss)

## Vision Layers

[tensorplay.nn.modules.pixelshuffle.PixelShuffle](generated/tensorplay.nn.modules.pixelshuffle.PixelShuffle.html#tensorplay.nn.modules.pixelshuffle.PixelShuffle)

Rearrange elements in a tensor according to an upscaling factor.

[tensorplay.nn.modules.pixelshuffle.PixelUnshuffle](generated/tensorplay.nn.modules.pixelshuffle.PixelUnshuffle.html#tensorplay.nn.modules.pixelshuffle.PixelUnshuffle)

Reverse the PixelShuffle operation.

[tensorplay.nn.modules.upsampling.Upsample](generated/tensorplay.nn.modules.upsampling.Upsample.html#tensorplay.nn.modules.upsampling.Upsample)

Upsamples a given multi-channel 1D (temporal), 2D (spatial) or 3D (volumetric) data.

[tensorplay.nn.modules.upsampling.UpsamplingNearest2d](generated/tensorplay.nn.modules.upsampling.UpsamplingNearest2d.html#tensorplay.nn.modules.upsampling.UpsamplingNearest2d)

Applies a 2D nearest neighbor upsampling to an input signal composed of several input channels.

[tensorplay.nn.modules.upsampling.UpsamplingBilinear2d](generated/tensorplay.nn.modules.upsampling.UpsamplingBilinear2d.html#tensorplay.nn.modules.upsampling.UpsamplingBilinear2d)

Applies a 2D bilinear upsampling to an input signal composed of several input channels.

## Shuffle Layers

[tensorplay.nn.modules.channelshuffle.ChannelShuffle](generated/tensorplay.nn.modules.channelshuffle.ChannelShuffle.html#tensorplay.nn.modules.channelshuffle.ChannelShuffle)

Divides and rearranges the channels in a tensor.

## DataParallel Layers (multi-GPU, distributed)

[tensorplay.nn.parallel.data_parallel.DataParallel](generated/tensorplay.nn.parallel.data_parallel.DataParallel.html#tensorplay.nn.parallel.data_parallel.DataParallel)

Implements data parallelism at the module level.

[tensorplay.nn.parallel.distributed.DistributedDataParallel](generated/tensorplay.nn.parallel.distributed.DistributedDataParallel.html#tensorplay.nn.parallel.distributed.DistributedDataParallel)

Implements distributed data parallelism (torch parity).

## Utilities

[tensorplay.nn.utils.rnn.PackedSequence](generated/tensorplay.nn.utils.rnn.PackedSequence.html#tensorplay.nn.utils.rnn.PackedSequence)

Holds the data and list of batch_sizes of a packed sequence.

[tensorplay.nn.utils.rnn.pack_padded_sequence](generated/tensorplay.nn.utils.rnn.pack_padded_sequence.html#tensorplay.nn.utils.rnn.pack_padded_sequence)

Packs a Tensor containing padded sequences of variable length.

[tensorplay.nn.utils.rnn.pad_packed_sequence](generated/tensorplay.nn.utils.rnn.pad_packed_sequence.html#tensorplay.nn.utils.rnn.pad_packed_sequence)

Pad a packed batch of variable length sequences.

[tensorplay.nn.utils.rnn.pad_sequence](generated/tensorplay.nn.utils.rnn.pad_sequence.html#tensorplay.nn.utils.rnn.pad_sequence)

Pad a list of variable length Tensors with padding_value.

[tensorplay.nn.utils.rnn.pack_sequence](generated/tensorplay.nn.utils.rnn.pack_sequence.html#tensorplay.nn.utils.rnn.pack_sequence)

Packs a list of variable length Tensors.

[tensorplay.nn.utils.rnn.unpack_sequence](generated/tensorplay.nn.utils.rnn.unpack_sequence.html#tensorplay.nn.utils.rnn.unpack_sequence)

Unpack PackedSequence into a list of variable length Tensors.

[tensorplay.nn.utils.rnn.unpad_sequence](generated/tensorplay.nn.utils.rnn.unpad_sequence.html#tensorplay.nn.utils.rnn.unpad_sequence)

Unpad padded Tensor into a list of variable length Tensors.

[tensorplay.nn.utils.rnn.invert_permutation](generated/tensorplay.nn.utils.rnn.invert_permutation.html#tensorplay.nn.utils.rnn.invert_permutation)

Returns the inverse of permutation.

[tensorplay.nn.parameter.is_lazy](generated/tensorplay.nn.parameter.is_lazy.html#tensorplay.nn.parameter.is_lazy)

Returns whether param is an UninitializedParameter or UninitializedBuffer.

[tensorplay.nn.factory_kwargs](generated/tensorplay.nn.factory_kwargs.html#tensorplay.nn.factory_kwargs)

Return a canonicalized dict of factory kwargs.

[tensorplay.nn.modules.flatten.Flatten](generated/tensorplay.nn.modules.flatten.Flatten.html#tensorplay.nn.modules.flatten.Flatten)

Flattens a contiguous range of dims into a tensor.

[tensorplay.nn.modules.flatten.Unflatten](generated/tensorplay.nn.modules.flatten.Unflatten.html#tensorplay.nn.modules.flatten.Unflatten)

Unflattens a tensor dim expanding it to a desired shape.

## Lazy Modules Initialization

[tensorplay.nn.modules.lazy.LazyModuleMixin](generated/tensorplay.nn.modules.lazy.LazyModuleMixin.html#tensorplay.nn.modules.lazy.LazyModuleMixin)

A mixin for modules that lazily initialize parameters, also known as "lazy modules".

## TensorPlay-specific additions

[Buffer](generated/tensorplay.nn.Buffer.html#tensorplay.nn.Buffer)

A kind of Tensor that should not be considered a model parameter.

[DepthwiseConv2d](generated/tensorplay.nn.DepthwiseConv2d.html#tensorplay.nn.DepthwiseConv2d)

[NonDynamicallyQuantizableLinear](generated/tensorplay.nn.NonDynamicallyQuantizableLinear.html#tensorplay.nn.NonDynamicallyQuantizableLinear)

[Parameter](generated/tensorplay.nn.Parameter.html#tensorplay.nn.Parameter)

A kind of Tensor that is to be considered a module parameter.

[RNNCellBase](generated/tensorplay.nn.RNNCellBase.html#tensorplay.nn.RNNCellBase)

[UninitializedBuffer](generated/tensorplay.nn.UninitializedBuffer.html#tensorplay.nn.UninitializedBuffer)

A buffer that is not initialized.

[UninitializedParameter](generated/tensorplay.nn.UninitializedParameter.html#tensorplay.nn.UninitializedParameter)

A parameter that is not initialized.