tensorplay.autograd.gradcheck.gradgradcheck

tensorplay.autograd.gradcheck.gradgradcheck(func, inputs, grad_outputs=None, *, eps: float = 1e-06, atol: float = 1e-05, rtol: float = 0.001, gen_non_contig_grad_outputs: bool = False, raise_exception: bool = True, nondet_tol: float = 0.0, check_undefined_grad: bool = True, check_grad_dtypes: bool = False, check_batched_grad: bool = False, check_fwd_over_rev: bool = False, check_rev_over_rev: bool = True, fast_mode: bool = False, masked: bool = False) → bool[source]

Check gradients of gradients computed via small finite differences against analytical gradients wrt tensors in inputs and grad_outputs that are of floating point or complex type and with requires_grad=True.

This function checks that backpropagating through the gradients computed to the given grad_outputs are correct.

The check between numerical and analytical gradients uses allclose().

Note

The default values are designed for input and grad_outputs of double precision. This check will likely fail if they are of less precision, e.g., FloatTensor.

Parameters:

• func (function) – a Python function that takes Tensor inputs and returns a Tensor or a tuple of Tensors

• inputs (tuple of Tensor or Tensor) – inputs to the function

• grad_outputs (tuple of [Tensor or None] or Tensor, optional) – The gradients with respect to the function’s outputs.

• eps (float, optional) – perturbation for finite differences

• atol (float, optional) – absolute tolerance

• rtol (float, optional) – relative tolerance

• gen_non_contig_grad_outputs (bool, optional) – Not supported by this engine yet; True raises NotImplementedError.

• raise_exception (bool, optional) – indicating whether to raise an exception if the check fails. The exception gives more information about the exact nature of the failure. This is helpful when debugging gradchecks.

• nondet_tol (float, optional) – tolerance for non-determinism. When running identical inputs through the differentiation, the results must either match exactly (default, 0.0) or be within this tolerance. Note that a small amount of nondeterminism in the gradient will lead to larger inaccuracies in the second derivative.

• check_undefined_grad (bool, optional) – if True, check if undefined output grads are supported and treated as zeros

• check_batched_grad (bool, optional) – Not supported by this engine yet.

• fast_mode (bool, optional) – Not supported by this engine yet.

• masked (bool, optional) – Kept for signature parity with torch.

Returns:

True if all differences satisfy allclose condition