Attention
=========

Computes scaled dot product attention on query, key and value tensors, using an optional attention mask if passed. This operator covers self and cross variants of the attention operation based on sequence lengths of K, Q and V. For self attention, `kv_sequence_length` equals to `q_sequence_length`. ...

**Abstract Signature:**

``Attention(Q, K, V, attn_mask, past_key, past_value, nonpad_kv_seqlen: int, is_causal: int, kv_num_heads: int, q_num_heads: int, qk_matmul_output_mode: int, scale: float, softcap: float, softmax_precision: int)``

.. raw:: html

 
    <div class="op-tabs-container"> 
      <div class="op-tabs-nav"> 
        <button class="op-tab-btn active" onclick="openOpTab(event, 'Keras_0')">Keras</button><button class="op-tab-btn " onclick="openOpTab(event, 'TensorFlow_1')">TensorFlow</button><button class="op-tab-btn " onclick="openOpTab(event, 'Apple MLX_2')">Apple MLX</button> 
      </div> 
      <div class="op-tabs-content"> 
        <div id="Keras_0" class="op-tab-pane active"><h4>Keras</h4><div class="op-detail-row"><span class="label">API:</span> <code>keras.layers.Attention</code></div><div class="op-detail-row"><span class="label">Strategy:</span> <span>Direct Mapping</span></div><div class="op-detail-row"><a href="https://keras.io/search.html?q=keras.layers.Attention" target="_blank" class="op-doc-link">Official Docs ↗</a></div></div><div id="TensorFlow_1" class="op-tab-pane "><h4>TensorFlow</h4><div class="op-detail-row"><span class="label">API:</span> <code>keras.layers.Attention</code></div><div class="op-detail-row"><span class="label">Strategy:</span> <span>Direct Mapping</span></div><div class="op-detail-row"><a href="https://www.tensorflow.org/api_docs/python/keras/layers/Attention" target="_blank" class="op-doc-link">Official Docs ↗</a></div></div><div id="Apple MLX_2" class="op-tab-pane "><h4>Apple MLX</h4><div class="op-detail-row"><span class="label">API:</span> <code>mlx.nn.layers.transformer.TransformerEncoderLayer.attention</code></div><div class="op-detail-row"><span class="label">Strategy:</span> <span>Direct Mapping</span></div><div class="op-detail-row"><a href="https://ml-explore.github.io/mlx/build/html/python/_autosummary/mlx.nn.layers.transformer.TransformerEncoderLayer.attention.html" target="_blank" class="op-doc-link">Official Docs ↗</a></div></div> 
      </div> 

      <!-- Load JS Logic only once per page ideally, but safe to exist globally --> 
      <script src="../_static/op_tabs.js"></script> 
    </div>