开发者
资源
[object Object]

[object Object][object Object]undefined
[object Object]
  • 接口功能:的反向传播。用于计算输入张量的梯度,以便在反向传播过程中更新模型参数。

  • 计算公式:

    正向:

    out=LayerNorm(input)×(1+scale)+shiftout = LayerNorm(input)\times(1+scale)+shift

    反向:

    z=(inputmean)×rstdz = (input - mean) \times rstd dy_g=gradOut×weightOptional×(1+scale)dy\_g = gradOut \times weightOptional \times (1 + scale) temp1=1/N×reduce_axis_2(gradOut×weightOptional×(1+scale))temp_1 = 1/N \times \sum_{reduce\_axis\_2} (gradOut \times weightOptional \times (1 + scale)) temp2=1/N×(inputmean)×rstd×reduce_axis_2(gradOut×weightOptional×(1+scale)×(inputmean)×rstd)temp_2 = 1/N \times (input - mean) \times rstd \times \sum_{reduce\_axis\_2}(gradOut \times weightOptional \times (1 + scale) \times (input - mean) \times rstd) gradInputOut=(gradOut×weightOptional×(1+scale)(temp1+temp2))×rstdgradInputOut = (gradOut \times weightOptional \times (1 + scale) - (temp_1 + temp_2)) \times rstd gradScaleOut=reduce_axis_1(gradOut×((inputmean)×rstd×weightOptional+biasOptional))gradScaleOut = \sum_{reduce\_axis\_1}(gradOut \times ((input - mean) \times rstd \times weightOptional + biasOptional)) gradShiftOut=reduce_axis_1gradOutgradShiftOut = \sum_{reduce\_axis\_1}gradOut gradWeightOut=reduce_axis_0,1gradOut×(1+scale)×(inputmean)×rstdgradWeightOut = \sum_{reduce\_axis\_0,1}gradOut \times (1 + scale) \times (input - mean) \times rstd gradBiasOut=reduce_axis_0,1gradOut×(1+scale)gradBiasOut = \sum_{reduce\_axis\_0,1}gradOut \times (1 + scale)

    其中,N为进行归一化计算的轴的维度,即归一化轴维度的大小。

[object Object]

每个算子分为,必须先调用“aclnnAdaLayerNormBackwardGetWorkspaceSize”接口获取计算所需workspace大小以及包含了算子计算流程的执行器,再调用“aclnnAdaLayerNormBackward”接口执行计算。

[object Object]
[object Object]
[object Object]
  • 参数说明

    [object Object]
  • 返回值

    aclnnStatus:返回状态码,具体参见

    第一段接口完成入参校验,出现以下场景时报错:

    [object Object]
[object Object]
  • 参数说明

    [object Object]
  • 返回值

    aclnnStatus:返回状态码,具体参见

[object Object]
  • 混合精度搭配:

    • AdaLayerNormBackward算子下列参数数据类型一致:

      [object Object]undefined
    • 上述两类数据类型固定搭配类型为(左侧和右侧数据类型严格对应):

      [object Object]undefined
  • 确定性计算:

    • aclnnAdaLayerNormBackward默认确定性实现。
[object Object]

示例代码如下,仅供参考,具体编译和执行过程请参考

[object Object]