[object Object]

[object Object][object Object]undefined[object Object]

• Description: Implements the AdamW optimizer.

• Formula:

  $$g_t=\begin{cases}-g_t & \text{ if } maxmize= true\\ g_t & \text{ if } maxmize=false \end{cases}$$ $$m_{t}=\beta_{1} m_{t-1}+\left(1-\beta_{1}\right) g_{t} \\$$ $$v_{t}=\beta_{2} v_{t-1}+\left(1-\beta_{2}\right) g_{t}^{2}$$ $$\beta_{1}^{t}=\beta_{1}^{t-1}\times\beta_{1}$$ $$\beta_{2}^{t}=\beta_{2}^{t-1}\times\beta_{2}$$ $$v_t=\begin{cases}\max(maxGradNorm, v_t) & \text{ if } amsgrad = true\\ v_t & \text{ if } amsgrad = false \end{cases}$$ $$\hat{m}_{t}=\frac{m_{t}}{1-\beta_{1}^{t}} \\$$ $$\hat{v}_{t}=\frac{v_{t}}{1-\beta_{2}^{t}} \\$$ $$\theta_{t+1}=\theta_{t}-\frac{\eta}{\sqrt{\hat{v}_{t}}+\epsilon} \hat{m}_{t}-\eta \cdot \lambda \cdot \theta_{t-1}$$

[object Object]

Each operator has calls. First, aclnnApplyAdamWGetWorkspaceSize is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, aclnnApplyAdamW is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters:

  • varRef (aclTensor*, compute input/compute output): weight input and output (theta in the formula), which is an aclTensor on the device. The shape support 1D to 8D. The data type can be FLOAT16, BFLOAT16, or FLOAT32. are supported. The can be ND.
  • mRef (aclTensor*, compute input/compute output): m parameter in the AdamW optimizer (m in the formula), which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape and dtype must be the same as those of varRef. are supported. The can be ND.
  • vRef (aclTensor*, compute input/compute output): v parameter in the AdamW optimizer (v in the formula), which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape and dtype must be the same as those of varRef. are supported. The can be ND.
  • beta1Power (aclTensor*, compute input): beta1^(t-1) parameter, which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape must be [1], and the dtype must be the same as that of varRef. are supported. The can be ND.
  • beta2Power (aclTensor*, compute input): beta2^(t-1) parameter, which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape must be [1], and the dtype must be the same as that of varRef. are supported. The can be ND.
  • lr (aclTensor*, compute input): learning rate (eta in the formula), which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape must be [1], and the dtype must be the same as that of varRef. are supported. The can be ND.
  • weightDecay (aclTensor*, compute input): weight decay coefficient, which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape must be [1], and the dtype must be the same as that of varRef. are supported. The can be ND.
  • beta1 (aclTensor*, compute input): beta1 parameter, which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape must be [1], and the dtype must be the same as that of varRef. are supported. The can be ND.
  • beta2 (aclTensor*, compute input): beta2 parameter, which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape must be [1], and the dtype must be the same as that of varRef. are supported. The can be ND.
  • eps (aclTensor*, compute input): parameter for avoiding division by zero, which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape must be [1], and the dtype must be the same as that of varRef. are supported. The can be ND.
  • grad (aclTensor*, compute input): gradient data (g_t in the formula), which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape and dtype must be the same as those of varRef. are supported. The can be ND.
  • maxGradNormOptional (aclTensor*, compute input): stores the maximum value of the v parameter (v in the formula), which is an aclTensor on the device. The data type can be FLOAT16, BFLOAT16, or FLOAT32. The shape and dtype must be the same as those of varRef. This parameter is mandatory when amsgrad is set to true and optional when amsgrad is set to false. are supported. The can be ND.
  • amsgrad (bool, compute input): whether to use the maxGradNormOptional variable. The data type is BOOL.
  • maximize (bool, compute input): whether to reverse the gradient. The gradient ascent direction is used to optimize the weight to maximize the loss function. The data type is BOOL.
  • workspaceSize (uint64_t*, output): size of the workspace to be allocated on the device.
  • executor (aclOpExecutor**, output): memory address.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• Parameters:

  • workspace (void *, input): address of the workspace to be allocated on the device.
  • workspaceSize (uint64_t, input): size of the workspace to be allocated on the device, which is obtained by calling the first-phase API aclnnApplyAdamWGetWorkspaceSize.
  • executor (aclOpExecutor *, input): operator executor, containing the operator computation process.
  • stream (aclrtStream, input): stream for executing the task.

• Returns:

  aclnnStatus: status code. For details, see .

[object Object]

• The data types of the input tensors must be the same. The data type can be FLOAT16, BFLOAT16, or FLOAT32.

• The shape sizes of the input tensors beta1Power, beta2Power, lr, weightDecay, beta1, beta2, and eps should be 1.

• When the input Boolean value of maximize is true, the maxGradNormOptional parameter is mandatory, and the data type and shape must be the same as those of varRef.

• Deterministic compute:

  • aclnnApplyAdamW defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .