# tensorplay.nn.functional.scaled_dot_product_attention

Source: https://www.tensorplay.cn/docs/generated/tensorplay.nn.functional.scaled_dot_product_attention.html

# tensorplay.nn.functional.scaled_dot_product_attention

tensorplay.nn.functional.scaled_dot_product_attention(query: TensorBase, key: TensorBase, value: TensorBase, attn_mask=None, dropout_p: [float](https://docs.python.org/3/library/functions.html#float) = 0.0, is_causal: [bool](https://docs.python.org/3/library/functions.html#bool) = False, scale=None, backend: [str](https://docs.python.org/3/library/stdtypes.html#str) | [None](https://docs.python.org/3/library/constants.html#None) = None) → TensorBase[[source]](../_modules/tensorplay/nn/functional.html#scaled_dot_product_attention)

scaled_dot_product_attention(query, key, value, attn_mask=None,
dropout_p=0.0, is_causal=False, scale=None, backend=None) -> Tensor

Computes scaled dot product attention on query, key and value. Routes to
the fused native kernel when possible, otherwise follows torch’s math
reference:

\[\text{Attention}(Q, K, V) = \text{softmax}(\frac{Q K^T}{\sqrt{E}}) V\]

Parameters:

backend ([str](https://docs.python.org/3/library/stdtypes.html#str), optional) – 'flash' | 'mem_efficient' |
'math', or None to pick automatically. 'flash'
selects the fused flash-attention kernel (impl=1), 'math'
forces the composed reference path; 'mem_efficient',
matching torch’s memory-efficient backend, is not available in
this build.