[object Object]

[object Object][object Object]undefined[object Object]

• Description:

  Collects the mean value and variance of data on all devices and updates the global mean value and the reciprocal of the standard deviation. The performance of BatchNorm is related to BatchSize. The larger the value of BatchSize is, the more accurate the statistics of BatchNorm will be. However, tasks such as detection occupy a large amount of Video RAM. A graphics card usually uses only a small number of images for training, for example, two images. As a result, the BatchNorm performance deteriorates. One solution is SyncBatchNorm, that is, all devices share the same BatchNorm to obtain the global statistics.

  During aclnnBatchNormGatherStatsWithCounts computation, aclnnBatchNormStats is required to compute the mean and reciprocal of the standard deviation of the data on a single card.

• Formula:

  $$y = \frac{(x-E[x])}{\sqrt{Var(x)+ eps}} * γ + β$$

  Where, the updated formulas for runningMean and runningVar are as follows:

  $$runningMean=runningMean*(1-momentum) + E[x]*momentum$$ $$runningVar=runningVar*(1-momentum) + Var[x]*momentum$$

[object Object]

Each operator has calls. First, aclnnBatchNormGatherStatsWithCountsGetWorkspaceSize is called to obtain the input parameters and compute the required workspace size based on the process. Then, aclnnBatchNormGatherStatsWithCounts is called to perform computation.

[object Object]

• Parameters

  [object Object]
  • [object Object]Atlas training products[object Object] and [object Object]Atlas inference products[object Object]: The data types of [object Object], [object Object], [object Object], [object Object], [object Object], [object Object], [object Object], and [object Object] cannot be BFLOAT16.

• Returns:

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns:

  [object Object]: status code. For details, see .

[object Object]

• Deterministic compute:
  • aclnnBatchNormGatherStatsWithCounts defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .