[object Object]

Note: This API will be deprecated in later versions. Use the latest API aclnnMlaPrologV3WeightNz instead.

[object Object][object Object]undefined[object Object]

• Interface function: In inference scenarios, this operator performs the preprocessing computation for Multi-Head Latent Attention. The computation process consists of four paths. First, the input $x$ is multiplied by $W^{DQ}$ for downsampling and RmsNorm, and then split into two paths. In the first path, $W^{UQ}$ and $W^{UK}$ are applied, followed by two upsampling operations to obtain $q^N$. In the second path, $W^{QR}$ is applied and rotary position encoding (ROPE) is used to obtain $q^R$. The third path multiplies input $x$ by $W^{DKV}$ for downsampling and RmsNorm, and the result is stored into the cache to obtain $k^C$. The fourth path multiplies input $x$ by $W^{KR}$, applies rotary position encoding, and then stores the result into another cache to obtain $k^R$.

• Formula:

  RmsNorm formula

  $$\text{RmsNorm}(x) = \gamma \cdot \frac{x_i}{\text{RMS}(x)}$$ $$\text{RMS}(x) = \sqrt{\frac{1}{N} \sum_{i=1}^{N} x_i^2 + \epsilon}$$

  The computation formula of the query, including downsampling, RmsNorm, and two upsampling operations.

  $$c^Q = RmsNorm(x \cdot W^{DQ})$$ $$q^C = c^Q \cdot W^{UQ}$$ $$q^N = q^C \cdot W^{UK}$$

  Performs rotary position encoding (ROPE) to the query.

  $$q^R = ROPE(c^Q \cdot W^{QR})$$

  The computation formula of the key, including downsampling and RmsNorm. The computation result is stored in the cache.

  $$c^{KV} = RmsNorm(x \cdot W^{DKV})$$ $$k^C = Cache(c^{KV})$$

  Performs rotary position encoding (ROPE) to the key and stores the result in the cache.

  $$k^R = Cache(ROPE(x \cdot W^{KR}))$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]undefined

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]undefined

[object Object]

• Parameters

  [object Object]undefined

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computing:

  • [object Object] defaults to deterministic implementation. The details are as follows:

• Shape field description

  [object Object]undefined

• When transposing is not performed, the dimensions of [object Object], [object Object], and [object Object] are represented as (k, n).

• The aclnnMlaProlog API supports the following scenarios for A2 and A3:

  [object Object]

• In different quantization scenarios, the combination of dtype and shape of parameters must meet the following conditions:

  [object Object][object Object]

[object Object]

The following example is for reference only. For details, see .