[object Object]

[object Object][object Object]undefined[object Object]

• API function: The TransformBiasRescaleQkv operator is an interface used to process the query, key, and value vectors in the multi-head attention mechanism. It is used to adjust the bias and rescale factors of these vectors to optimize the attention computation process.

• Formulas: The element-wise computation process is as follows:

  $$q_o=(q_i+q_{bias})/\sqrt{dim\_per\_head}\\$$ $$k_o=k_i+k_{bias}\\$$ $$v_o=v_i+v_{bias}$$

  In the formula:

  • [object Object] indicates the dimension of each attention head.
  • q[object Object]o[object Object], k[object Object]o[object Object], and v[object Object]o[object Object] are the output elements of the query, key, and value vectors, respectively.
  • q[object Object]i[object Object], k[object Object]i[object Object], and v[object Object]i[object Object] are the input elements of the query, key, and value vectors, respectively.
  • q[object Object]bias[object Object], k[object Object]bias[object Object], and v[object Object]bias[object Object] are the input element offsets of the query, key, and value vectors, respectively.

[object Object]

Each operator has calls. First, [object Object] is called to obtain the input parameters and compute the required workspace size based on the process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter validation. The following error codes may be returned.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computation:

  • [object Object] defaults to a deterministic implementation.

• The data types of the inputs [object Object] and [object Object], and outputs [object Object], [object Object], and [object Object] must be the same.

• If the input is NaN, the output is also NaN. If the input is Inf, the output is also Inf.

• If the input is [object Object], the output is [object Object].

[object Object]

The following example is for reference only. For details, see .