aclnnQuantizedBatchNorm
Applicable Products
| Product | Supported |
|---|---|
| √ | |
| √ | |
| √ | |
| × | |
| × | |
| × |
Function
Description:
Performs dequantization on the input tensor, then performs normalization based on the input weight, bias, and epsilon, and finally performs quantization based on the output outputScale and outputZeroPoint.
Formula:
- Dequantization calculation:
- Normalization calculation:
- Quantization calculation:
Function Prototype
Each operator is divided into a two-phase API. You must first call the "aclnnQuantizedBatchNormGetWorkspaceSize" API to obtain the input parameters and calculate the required workspace size based on the computation process, and then call the "aclnnQuantizedBatchNorm" API to perform the computation.
aclnnStatus aclnnQuantizedBatchNormGetWorkspaceSize(
const aclTensor* input,
const aclTensor* mean,
const aclTensor* var,
const aclScalar* inputScale,
const aclScalar* inputZeroPoint,
const aclScalar* outputScale,
const aclScalar* outputZeroPoint,
aclTensor* weight,
aclTensor* bias,
float epsilon,
aclTensor* output,
uint64_t *workspaceSize,
aclOpExecutor **executor)aclnnStatus aclnnQuantizedBatchNorm(
void *workspace,
uint64_t workspaceSize,
aclOpExecutor *executor,
aclrtStream stream)aclnnQuantizedBatchNormGetWorkspaceSize
Parameters
Parameter Input/Output Description Instruction Data Type Data Format Dimension (Shape) Non-contiguous Tensor input (aclTensor*) Input Indicates the quantized data input to the model, corresponding to `x` in the formula. Does not support null tensor. INT8, UINT8, INT32 NCHW 4 √ mean (aclTensor*) Input Indicates the mean of the model input data, corresponding to `mean` in the formula. - Does not support null tensor.
- The shape length is equal to the channel axis length in the input parameter
input.
FLOAT32 ND 1 √ var (aclTensor*) Input Indicates the variance of the model input data, corresponding to `var` in the formula. - Does not support null tensors.
- The shape length is equal to the length of the channel axis in the input parameter
input.
FLOAT32 ND 1 √ inputScale (aclScalar*) Input Indicates the scale factor of the input data, corresponding to `inputScale` in the formula. - FLOAT32 ND - - inputZeroPoint (aclScalar*) Input Indicates the bias of the model input data, corresponding to `inputZeroPoint` in the formula. The passed value must not exceed the upper and lower bounds of the data type corresponding to `input`. For example, the upper and lower bounds of INT8 are [-128, 127]. INT32 ND - - outputScale (aclScalar*) Input Indicates the scale factor of the model output data, corresponding to `outputScale` in the formula. - FLOAT32 ND - - outputZeroPoint (aclScalar*) Input Indicates the offset of the model output data, corresponding to `outputZeroPoint` in the formula. The input value cannot exceed the upper and lower bounds of the data type corresponding to input. For example, the upper and lower bounds of INT8 are [-128, 127].INT32 ND - - weight (aclTensor*) Input Optional parameter. Indicates the normalization weight, corresponding to `weight` in the formula. - Does not support null tensor.
- Default value: 1.
- The shape length is equal to the channel axis length of the input parameter `input`.
FLOAT32 ND 1 √ bias (aclTensor*) Input Optional parameter, indicates the normalization bias, corresponding to `bias` in the formula. - Does not support null tensor.
- Default value: 0.
- The shape length is equal to the channel axis length of the input `input`.
FLOAT32 ND 1 √ epsilon (float) Input Indicates the value added to the variance to avoid division by zero. Corresponds to `epsilon` in the formula. - - - - - output (aclTensor*) Output Indicates the quantized data output by the model, corresponding to `output` in the formula. - Does not support null tensor.
- The shape, data format, and data type are consistent with the input `input`.
INT8, UINT8, INT32 NCHW 4 √ workspaceSize (uint64_t*) Output Returns the workspace size to be applied on the Device side. - - - - - executor (aclOpExecutor**) Output Returns the op executor, which contains the operator computation flow. - - - - - Return Value
aclnnStatus: return code. For details, see aclnn Return Code.
The first-phase API performs input parameter validation and returns an error in the following scenarios:
Return Code Error Code Description ACLNN_ERR_PARAM_NULLPTR 161001 The input or output is a null pointer. ACLNN_ERR_PARAM_INVALID 161002 The data type is not within the supported range, or the parameters do not meet the above constraints.
aclnnQuantizedBatchNorm
Parameters
Parameter Input/Output Description workspace Input Memory address of the workspace applied on the device side. workspaceSize Input Size of the workspace applied on the device side, obtained from the first-phase API aclnnQuantizedBatchNormGetWorkspaceSize. executor Input Op executor, which contains the operator computation flow. stream Input Specifies the stream for task execution. Return Value
aclnnStatus: return code. For details, see aclnn Return Code.
Constraints
Deterministic computation:
- aclnnQuantizedBatchNorm defaults to a deterministic implementation.
Example
The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.
#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_quantized_batch_norm.h"
#define CHECK_RET(cond, return_expr) \
do { \
if (!(cond)) { \
return_expr; \
} \
} while (0)
#define LOG_PRINT(message, ...) \
do { \
printf(message, ##__VA_ARGS__); \
} while (0)
int64_t GetShapeSize(const std::vector<int64_t>& shape)
{
int64_t shapeSize = 1;
for (auto i : shape) {
shapeSize *= i;
}
return shapeSize;
}
int Init(int32_t deviceId, aclrtStream* stream)
{
// Boilerplate: resource initialization.
auto ret = aclInit(nullptr);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
ret = aclrtSetDevice(deviceId);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
ret = aclrtCreateStream(stream);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
return 0;
}
template <typename T>
int CreateAclTensor(
const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr, aclDataType dataType,
aclTensor** tensor)
{
auto size = GetShapeSize(shape) * sizeof(T);
// Call aclrtMalloc to apply for memory on the device side.
auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
// Copy host-side data to device-side memory using aclrtMemcpy.
ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);
// Compute the strides of a contiguous tensor.
std::vector<int64_t> strides(shape.size(), 1);
for (int64_t i = shape.size() - 2; i >= 0; i--) {
strides[i] = shape[i + 1] * strides[i + 1];
}
// Create an aclTensor using the aclCreateTensor API.
*tensor = aclCreateTensor(
shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(),
*deviceAddr);
return 0;
}
template <typename T>
int CreateAclTensorNCHW(
const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr, aclDataType dataType,
aclTensor** tensor)
{
auto size = GetShapeSize(shape) * sizeof(T);
// Allocate device-side memory using aclrtMalloc.
auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
// Copy host-side data to device-side memory using aclrtMemcpy.
ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);
// Compute the strides of a contiguous tensor.
std::vector<int64_t> strides(shape.size(), 1);
for (int64_t i = shape.size() - 2; i >= 0; i--) {
strides[i] = shape[i + 1] * strides[i + 1];
}
// Call the aclCreateTensor API to create an aclTensor.
*tensor = aclCreateTensor(
shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_NCHW, shape.data(), shape.size(),
*deviceAddr);
return 0;
}
int main()
{
// 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
// Fill in the deviceId based on your actual device.
int32_t deviceId = 0;
aclrtStream stream;
auto ret = Init(deviceId, &stream);
// Handle the check based on your needs.
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);
// 2. Construct the input and output. Customize the construction based on the API interface.
std::vector<int64_t> selfShape = {1, 2, 1, 4};
std::vector<int64_t> meanShape = {2};
std::vector<int64_t> varShape = {2};
std::vector<int64_t> weightShape = {2};
std::vector<int64_t> biasShape = {2};
std::vector<int64_t> outShape = {1, 2, 1, 4};
void* selfDeviceAddr = nullptr;
void* weightDeviceAddr = nullptr;
void* biasDeviceAddr = nullptr;
void* rMeanDeviceAddr = nullptr;
void* rVarDeviceAddr = nullptr;
void* outDeviceAddr = nullptr;
void* meanDeviceAddr = nullptr;
void* varDeviceAddr = nullptr;
aclTensor* self = nullptr;
aclTensor* weight = nullptr;
aclTensor* bias = nullptr;
aclTensor* out = nullptr;
aclTensor* mean = nullptr;
aclTensor* var = nullptr;
std::vector<int32_t> selfHostData = {0, 1, 2, 3, 4, 5, 6, 7};
std::vector<float> weightHostData = {1, 1};
std::vector<float> biasHostData = {0, 0};
std::vector<float> meanHostData = {0, 0};
std::vector<float> varHostData = {1, 1};
std::vector<int32_t> outHostData(8, 0);
float inputScaleValue = 1.0f;
int32_t inputZeroPointValue = 1;
float outputScaleValue = 2.0f;
int32_t outputZeroPointValue = 1;
double eps = 1e-5;
// Create the self aclTensor.
ret = CreateAclTensorNCHW(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_INT32, &self);
CHECK_RET(ret == ACL_SUCCESS, return ret);
// Create the mean aclTensor.
ret = CreateAclTensor(meanHostData, meanShape, &meanDeviceAddr, aclDataType::ACL_FLOAT, &mean);
CHECK_RET(ret == ACL_SUCCESS, return ret);
// Create the var aclTensor.
ret = CreateAclTensor(varHostData, varShape, &varDeviceAddr, aclDataType::ACL_FLOAT, &var);
CHECK_RET(ret == ACL_SUCCESS, return ret);
// Create the weight aclTensor.
ret = CreateAclTensor(weightHostData, weightShape, &weightDeviceAddr, aclDataType::ACL_FLOAT, &weight);
CHECK_RET(ret == ACL_SUCCESS, return ret);
// Create the bias aclTensor.
ret = CreateAclTensor(biasHostData, biasShape, &biasDeviceAddr, aclDataType::ACL_FLOAT, &bias);
CHECK_RET(ret == ACL_SUCCESS, return ret);
// Create the out aclTensor.
ret = CreateAclTensorNCHW(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_INT32, &out);
CHECK_RET(ret == ACL_SUCCESS, return ret);
// Create the inputScale aclScalar.
aclScalar* inputScale = aclCreateScalar(&inputScaleValue, aclDataType::ACL_FLOAT);
// Create the inputZeroPoint aclScalar.
aclScalar* inputZeroPoint = aclCreateScalar(&inputZeroPointValue, aclDataType::ACL_INT32);
// Create the outputScale aclScalar.
aclScalar* outputScale = aclCreateScalar(&outputScaleValue, aclDataType::ACL_FLOAT);
// Create the outputZeroPoint aclScalar.
aclScalar* outputZeroPoint = aclCreateScalar(&outputZeroPointValue, aclDataType::ACL_INT32);
uint64_t workspaceSize = 0;
aclOpExecutor* executor;
// Example of aclnnQuantizedBatchNorm API call
// 3. Call the CANN operator library API. Replace with the specific API name.
// Call the first-phase API of aclnnQuantizedBatchNorm.
ret = aclnnQuantizedBatchNormGetWorkspaceSize(
self, mean, var, inputScale, inputZeroPoint, outputScale, outputZeroPoint, weight, bias, eps, out,
&workspaceSize, &executor);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnQuantizedBatchNormGetWorkspaceSize failed. ERROR: %d\n", ret);
return ret);
// Apply for device memory based on the workspaceSize calculated by the first-phase API.
void* workspaceAddr = nullptr;
if (workspaceSize > 0) {
ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
}
// Call the second-phase API of aclnnQuantizedBatchNorm.
ret = aclnnQuantizedBatchNorm(workspaceAddr, workspaceSize, executor, stream);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnQuantizedBatchNorm failed. ERROR: %d\n", ret); return ret);
// 4. (Boilerplate) Wait synchronously for task execution to complete.
ret = aclrtSynchronizeStream(stream);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);
// 5. Obtain the output value and copy the result from the Device side memory to the Host side. Modify based on the specific API definition.
auto size = GetShapeSize(outShape);
printf("size is %ld", size);
std::vector<int32_t> resultData(size, 0);
ret = aclrtMemcpy(
resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr, size * sizeof(resultData[0]),
ACL_MEMCPY_DEVICE_TO_HOST);
CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
for (int64_t i = 0; i < size; i++) {
LOG_PRINT("result[%ld] is: %d\n", i, resultData[i]);
}
// 6. Release aclTensor and aclScalar. Modify based on the specific API definition.
aclDestroyTensor(self);
aclDestroyTensor(weight);
aclDestroyTensor(bias);
aclDestroyTensor(mean);
aclDestroyTensor(var);
aclDestroyTensor(out);
aclDestroyScalar(inputScale);
aclDestroyScalar(inputZeroPoint);
aclDestroyScalar(outputScale);
aclDestroyScalar(outputZeroPoint);
// 7. Release Device resources. Modify based on the specific API definition.
aclrtFree(selfDeviceAddr);
aclrtFree(meanDeviceAddr);
aclrtFree(varDeviceAddr);
aclrtFree(weightDeviceAddr);
aclrtFree(biasDeviceAddr);
aclrtFree(outDeviceAddr);
if (workspaceSize > 0) {
aclrtFree(workspaceAddr);
}
aclrtDestroyStream(stream);
aclrtResetDevice(deviceId);
aclFinalize();
return 0;
}