# tensorplay.nn.functional.multi_head_attention_forward

Source: https://www.tensorplay.cn/docs/generated/tensorplay.nn.functional.multi_head_attention_forward.html

# tensorplay.nn.functional.multi_head_attention_forward

tensorplay.nn.functional.multi_head_attention_forward(query: TensorBase, key: TensorBase, value: TensorBase, embed_dim_to_check: [int](https://docs.python.org/3/library/functions.html#int), num_heads: [int](https://docs.python.org/3/library/functions.html#int), in_proj_weight: TensorBase | [None](https://docs.python.org/3/library/constants.html#None) = None, in_proj_bias: TensorBase | [None](https://docs.python.org/3/library/constants.html#None) = None, bias_k=None, bias_v=None, add_zero_attn: [bool](https://docs.python.org/3/library/functions.html#bool) = False, dropout_p: [float](https://docs.python.org/3/library/functions.html#float) = 0.0, out_proj_weight: TensorBase | [None](https://docs.python.org/3/library/constants.html#None) = None, out_proj_bias: TensorBase | [None](https://docs.python.org/3/library/constants.html#None) = None, training: [bool](https://docs.python.org/3/library/functions.html#bool) = True, key_padding_mask=None, need_weights: [bool](https://docs.python.org/3/library/functions.html#bool) = True, attn_mask=None, use_separate_proj_weight: [bool](https://docs.python.org/3/library/functions.html#bool) = False, q_proj_weight=None, k_proj_weight=None, v_proj_weight=None, static_k=None, static_v=None, average_attn_weights: [bool](https://docs.python.org/3/library/functions.html#bool) = True, is_causal: [bool](https://docs.python.org/3/library/functions.html#bool) = False)[[source]](../_modules/tensorplay/nn/functional.html#multi_head_attention_forward)

torch-compatible multi_head_attention_forward.

Follows the structure of
third_party/pytorch/aten/src/ATen/native/transformers/attention.cpp
_scaled_dot_product_attention paths composed from dispatched primitives.