[object Object]

[object Object][object Object]undefined[object Object]

• Description: The m and v inputs of the optimizer are used as indexes to obtain their values in the qmap, and the values are multiplied by the absmax corresponding to each blockSize for dequantization. Then, the AdamW optimizer is implemented. The maximum value of the updated m and v is selected from each blockSize. Each blockSize of m and v corresponds to an absmax, and normalization is performed once. The indexes in the qmap corresponding to m and v are found using the binary search method as the output. The absmax is also used as the input for the next round of quantization.

• Optimizer calculation formula:

  $$m_{t}=\beta_{1} m_{t-1}+\left(1-\beta_{1}\right) g_{t} \\$$ $$v_{t}=\beta_{2} v_{t-1}+\left(1-\beta_{2}\right) g_{t}^{2}$$ $$\hat{m}_{t}=\frac{m_{t}}{1-\beta_{1}^{t}} \\$$ $$\hat{v}_{t}=\frac{v_{t}}{1-\beta_{2}^{t}} \\$$ $$\theta_{t+1}=\theta_{t}-\frac{\eta}{\sqrt{\hat{v}_{t}}+\epsilon} \hat{m}_{t}-\eta \cdot \lambda \cdot \theta_{t-1}$$

[object Object]

Each operator has calls. First, aclnnApplyAdamWQuantGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnApplyAdamWQuant is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters:

  • varRef (const aclTensor*, compute input/compute output): weight input and output (theta in the formula), which is an aclTensor on the device. The data type can be FLOAT, FLOAT16, or BFLOAT16. are not supported. The can be ND.
  • grad (aclTensor*, compute input): gradient (gt in the formula), which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32, which are the same as that of varRef. The shape must be the same as that of varRef. are not supported. The can be ND.
  • mRef (aclTensor*, compute input/compute output): index value of the m parameter in the AdamW optimizer formula before quantization, which is an aclTensor on the device. The specific value in qmapM is exported based on the index. The data type can be UINT8. The shape must be the same as that of varRef. are not supported. The can be ND.
  • vRef (aclTensor*, compute input/compute output): index value of the v parameter in the AdamW optimizer formula before quantization, which is an aclTensor on the device. The specific value in qmapV is exported based on the index. The data type can be UINT8. The shape must be the same as that of varRef. are not supported. The can be ND.
  • qmapM (aclTensor*, compute input): aclTensor on the device. The quantization mapping table is sorted in ascending order. The data type can be FLOAT32. The data format must be ND. The shape must be [256,]. are not supported. The can be ND.
  • qmapV (aclTensor*, compute input): aclTensor on the device. The quantization mapping table is sorted in ascending order. The data type can be FLOAT32. The data format must be ND. The shape must be [256,]. are not supported. The can be ND.
  • absmaxMRef (aclTensor*, compute input/compute output): aclTensor on the device. The compute input is percentile denormalization (multiplied by absmaxMRef) in the current dequantization phase. The compute output is the parameters for the current quantization and the next round of dequantization. The data type can be FLOAT32. The shape requirement is that every 256 m values correspond to one maximum value. The shape requirement is "absmaxMRef.size = m.size/blockSize". are not supported. The can be ND.
  • absmaxVRef (aclTensor*, compute input/compute output): aclTensor on the device. The compute input is percentile denormalization (multiplied by absmaxVRef) in the current dequantization phase. The compute output is the parameters for the current quantization and the next round of dequantization. The data type can be FLOAT32. The shape requirement is that every 256 v values correspond to one maximum value. The shape requirement is "absmaxVRef.size = m.size/blockSize". are not supported. The can be ND.
  • step (aclTensor*): number of iterations (t in the formula), which is an aclTensor on the device. The data type can be INT64. The shape is [1,]. The can be ND.
  • lr (float*, compute input): learning rate (eta in the formula). The recommended value is 1e-3, 1e-5, or 1e-8. The value ranges from 0 to 1. The data type can be float.
  • beta1 (float*, compute input): beta1 parameter in the AdamW optimizer formula. The recommended value is 0.9. The value ranges from 0 to 1. The data type can be float.
  • beta2 (float*, compute input): beta2 parameter in the AdamW optimizer formula. The recommended value is 0.99. The value ranges from 0 to 1. The data type can be float.
  • weightDecay (float*, compute input): weight decay coefficient (lambda in the AdamW optimizer formula). The recommended value is 0.999. The value ranges from 0 to 1. The data type can be float.
  • eps (float*, compute input): epsilon parameter in the AdamW optimizer formula, which is added to the denominator to prevent division by zero. The recommended value is 1e-8. The data type can be float.
  • gnormScale (float*, compute input): parameter for scaling the input parameter grad. The recommended value is 0.999. The value ranges from 0 to 1. The data type can be float.
  • blockSize (int64*, compute input): size of each block involved in computation. The value is fixed at 256. The data type can be int64.
  • quantModeOptional (char*, compute input): reserved parameter.
  • workspaceSize (uint64_t*, output): size of the workspace to be allocated on the NPU device.
  • executor (aclOpExecutor**, output): operator executor, containing the operator computation process.

• Returns:

  aclnnStatus: status code. For details, see .

  [object Object]

[object Object]

• Parameters:

  • workspace (void*, input): address of the workspace to be allocated on the device.
  • workspaceSize (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API aclnnApplyAdamWQuantGetWorkspaceSize.
  • executor (aclOpExecutor *, input): operator executor, containing the operator computation process.
  • stream (aclrtStream, input): stream for executing the task.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

The shape of varRef must meet the following constraints:

• varRef.shape = grad.shape

• varRef.shape = mRef.shape

• varRef.shape = vRef.shape

• varRef.size/blockSize = absmaxMRef.size

• varRef.size/blockSize = absmaxVRef.size

• Deterministic compute:

  • aclnnApplyAdamWQuant defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .