tensorplay.autograd

tensorplay.autograd 提供了实现任意标量值函数自动微分的类和函数。

它只需要对现有代码进行极少的更改 - 你只需要使用 requires_grad=True 关键字声明应计算梯度的 Tensor。 目前，我们仅支持浮点 Tensor 类型（half, float, double 和 bfloat16）和复数 Tensor 类型（cfloat, cdouble）的自动微分。

Classes

class Function [source]

Function()

Records operation history and defines formulas for differentiating ops.

Methods

apply(*args, **kwargs) [source]

---

backward(ctx, *grad_outputs) [source]

定义微分操作的公式。

---

forward(ctx, *args, **kwargs) [source]

执行操作。

---

class enable_grad [source]

enable_grad(orig_func=None)

Bases: _NoParamDecoratorContextManager

启用梯度计算的上下文管理器。

如果通过 ~no_grad 或 ~set_grad_enabled 禁用了梯度计算，则启用它。

此上下文管理器是线程局部的；它不会影响其他线程中的计算。

也可以作为装饰器使用。

INFO

enable_grad 是几种可以在局部启用或禁用梯度的机制之一，有关它们的比较，请参阅 locally-disable-grad-doc。

INFO

此 API 不适用于 forward-mode AD <forward-mode-ad>。

Example

# xdoctest: +SKIP
x = tensorplay.tensor([1.], requires_grad=True)
with tensorplay.no_grad():
    with tensorplay.enable_grad():
        y = x * 2
y.requires_grad
True
y.backward()
x.grad
tensor([2.])
@tensorplay.enable_grad()
def doubler(x):
    return x * 2
with tensorplay.no_grad():
    z = doubler(x)
z.requires_grad
True
@tensorplay.enable_grad()
def tripler(x):
    return x * 3
with tensorplay.no_grad():
    z = tripler(x)
z.requires_grad
True

Methods

clone(self) [source]

---

class no_grad [source]

no_grad() -> None

Bases: _NoParamDecoratorContextManager

Context-manager that disables gradient calculation.

Disabling gradient calculation is useful for inference, when you are sure that you will not call Tensor.backward(). It will reduce memory consumption for computations that would otherwise have requires_grad=True.

In this mode, the result of every computation will have requires_grad=False, even when the inputs have requires_grad=True. There is an exception! All factory functions, or functions that create a new Tensor and take a requires_grad kwarg, will NOT be affected by this mode.

This context manager is thread local; it will not affect computation in other threads.

Also functions as a decorator.

INFO

No-grad is one of several mechanisms that can enable or disable gradients locally see locally-disable-grad-doc for more information on how they compare.

INFO

This API does not apply to forward-mode AD <forward-mode-ad>. If you want to disable forward AD for a computation, you can unpack your dual tensors.

Example

x = tensorplay.tensor([1.], requires_grad=True)
with tensorplay.no_grad():
    y = x * 2
y.requires_grad
False
@tensorplay.no_grad()
def doubler(x):
    return x * 2
z = doubler(x)
z.requires_grad
False
@tensorplay.no_grad()
def tripler(x):
    return x * 3
z = tripler(x)
z.requires_grad
False
# factory function exception
with tensorplay.no_grad():
    a = tensorplay.nn.Parameter(tensorplay.rand(10))
a.requires_grad
True

Methods

__init__(self) -> None [source]

Initialize self. See help(type(self)) for accurate signature.

---

clone(self) [source]

---

class set_grad_enabled [source]

set_grad_enabled(mode: bool) -> None

Bases: _DecoratorContextManager

Context-manager that sets gradient calculation on or off.

set_grad_enabled will enable or disable grads based on its argument mode. It can be used as a context-manager or as a function.

This context manager is thread local; it will not affect computation in other threads.

Args

• mode (bool): Flag whether to enable grad (True), or disable (False). This can be used to conditionally enable gradients.

INFO

set_grad_enabled is one of several mechanisms that can enable or disable gradients locally see locally-disable-grad-doc for more information on how they compare.

INFO

This API does not apply to forward-mode AD <forward-mode-ad>.

Example

# xdoctest: +SKIP
x = tensorplay.tensor([1.], requires_grad=True)
is_train = False
with tensorplay.set_grad_enabled(is_train):
    y = x * 2
y.requires_grad
False
_ = tensorplay.set_grad_enabled(True)
y = x * 2
y.requires_grad
True
_ = tensorplay.set_grad_enabled(False)
y = x * 2
y.requires_grad
False

Methods

__init__(self, mode: bool) -> None [source]

Initialize self. See help(type(self)) for accurate signature.

---

clone(self) -> 'set_grad_enabled' [source]

Create a copy of this class

---

Functions

grad() [source]

grad(outputs: Union[tensorplay._C.TensorBase, collections.abc.Sequence[tensorplay._C.TensorBase]], inputs: Union[tensorplay._C.TensorBase, collections.abc.Sequence[tensorplay._C.TensorBase]], grad_outputs: Union[tensorplay._C.TensorBase, collections.abc.Sequence[tensorplay._C.TensorBase], NoneType] = None, retain_graph: Optional[bool] = None, create_graph: bool = False, allow_unused: Optional[bool] = None) -> tuple[tensorplay._C.TensorBase, ...]

计算并返回输出相对于输入的梯度之和。

grad_outputs 应该是长度与 output 匹配的序列，包含向量-雅可比积中的“向量”，通常是相对于每个输出的预计算梯度。如果输出不需要梯度 (require_grad)，则梯度可以是 None。

INFO

如果你在用户指定的 CUDA 流上下文中运行任何前向操作、创建 grad_outputs 和/或调用 grad，请参阅 Stream semantics of backward passes<bwd-cuda-stream-semantics>。

Args

• outputs (Tensor 或 GradientEdge 的序列): 微分函数的输出。
• inputs (Tensor 或 GradientEdge 的序列): 将返回其梯度的输入（并且不会累积到 .grad 中）。
• grad_outputs (Tensor 的序列): 向量-雅可比积中的“向量”。通常是相对于每个输出的梯度。对于标量张量或不需要梯度的张量，可以指定 None 值。如果所有 grad_tensors 都可以接受 None 值，则此参数是可选的。默认值: None。
• retain_graph (bool, optional): 如果为 False，用于计算梯度的图将被释放。请注意，在几乎所有情况下，都不需要将此选项设置为 True，通常可以以更有效的方式解决。默认为 create_graph 的值。
• create_graph (bool, optional): 如果为 True，将构建导数图，允许计算高阶导数乘积。
• Default: False。
• allow_unused (Optional[bool], optional): 如果为 False，指定在计算输出时未使用的输入（因此它们的梯度始终为零）是一个错误。默认为 materialize_grads 的值。

is_grad_enabled() [source]

is_grad_enabled()