[object Object]

[object Object][object Object]undefined[object Object]

• Description: Performs two small matrix multiplications on the input matrix x: it right-multiplies x by kroneckerP2 and left-multiplies the result by kroneckerP1, then applies per-token quantization to the matrix multiplication result.

• Formula:

  1. Right-multiply x by kroneckerP2:
  $$x' = x @ kroneckerP2$$
  2. Left-multiply x' by kroneckerP1:
  $$x'' = kroneckerP1@x'$$
  3. Compute the maximum absolute value along dimension 0 of x'' and divide by (7/clipRatio) to obtain the quantization factor for INT4 quantization:
  $$quantScale = [max(abs(x''[0,:,:])),max(abs(x''[1,:,:])),...,max(abs(x''[K,:,:]))]/(7 / clipRatio)$$
  4. Compute the output out:
  $$out = x'' / quantScale$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation flow. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  aclnnStatus: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  aclnnStatus: status code. For details, see .

[object Object]

• Deterministic computation:
  • aclnnFlatQuant defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .