MultiHeadAttention

Computes Multi-Head Attention.

Abstract Signature:

MultiHeadAttention(embed_dim: int, num_heads: int)

PyTorch

API: torch.nn.MultiheadAttention
Strategy: Direct Mapping
Official Docs ↗

Keras

API: keras.layers.MultiHeadAttention
Strategy: Direct Mapping
Official Docs ↗

TensorFlow

API: tf.keras.layers.MultiHeadAttention
Strategy: Direct Mapping
Official Docs ↗

Apple MLX

API: mlx.nn.MultiHeadAttention
Strategy: Direct Mapping
Official Docs ↗

Flax NNX

API: flax.nnx.MultiHeadAttention
Strategy: Direct Mapping
Official Docs ↗

PaxML / Praxis

API: praxis.layers.MultiHeadAttention
Strategy: Direct Mapping
Official Docs ↗