aclnnFakeQuantPerChannelAffineCachemask

Applicable Products

Product	Supported
Ascend 950PR/Ascend 950DT	×
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	×
Atlas inference products	×
Atlas training products	×

Function

Description: For the input data self, performs fake quantization on the input self along the specified axis using scale and zero_point, updates the value range of the fake quantization output based on quant_min and quant_max, and finally returns the result out and the corresponding position mask.
Formula: First compute the temporary variable qval based on the operator function, then compute out and mask.
- The data type of zero_point is int32.
$qval = Round(std::nearby\_int(self / scale) + zero\_point)$
- The data type of zero_point is float16 or float32.
$qval = std::lrintf(self / scale + zero\_point)$ $out = (Min(quant\_max, Max(quant\_min, qval)) - zero\_point) * scale$ $mask = (qval >= quant\_min) \& (qval <= quant\_max)$

Function Prototype

Each operator is divided into a two-phase API. You must first call the "aclnnFakeQuantPerChannelAffineCachemaskGetWorkspaceSize" API to obtain the input parameters and calculate the required workspace size based on the process, and then call the "aclnnFakeQuantPerChannelAffineCachemask" API to perform the computation.

aclnnStatus aclnnFakeQuantPerChannelAffineCachemaskGetWorkspaceSize(
  const aclTensor   *self,
  const aclTensor   *scale,
  const aclTensor   *zeroPoint,
  int64_t            axis,
  int64_t            quantMin,
  int64_t            quantMax,
  aclTensor         *out,
  aclTensor         *mask,
  uint64_t          *workspaceSize,
  aclOpExecutor     **executor)

aclnnStatus aclnnFakeQuantPerChannelAffineCachemask(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnFakeQuantPerChannelAffineCachemaskGetWorkspaceSize

Parameters:

Parameter	Input/Output	Description	Instruction	Data Type	Data Format	Dimension (Shape)	Non-Contiguous Tensor
self	Input	self in the formula.	-	FLOAT16, FLOAT32	ND	0-8	√
scale	Input	scale in the formula, indicates the scale factor for input fake quantization.	The size must be consistent with zeroPoint and with the size of self along the axis.	FLOAT16, FLOAT32	ND	1	√
zeroPoint	Input	zero_point in the formula, which indicates the zero-reference parameter for fake quantization of the input.	The size must be consistent with that of scale and with that of self along the axis dimension.	INT32	ND	1	√
axis	Input	Indicates the dimension along which the computation is performed.	The value range is [-self.dim(), self.dim()-1].	INT64	-	-	-
quantMin	Input	Indicates the minimum value after fake quantization of the input data.	Must be less than or equal to quantMax.	INT64	-	-	-
quantMax	Input	Indicates the maximum value of the input data after fake quantization.	Must be greater than or equal to quantMin.	INT64	-	-	-
out	Output	out in the formula.	The shape must be consistent with `self`.	FLOAT16、FLOAT32	ND	0-8	√
mask	Output	Mask in the formula.	The shape must be consistent with `self`.	BOOL	ND	0-8	√
workspaceSize	Output	Returns the workspace size to be applied on the Device side.	-	-	-	-	-
executor	Output	Returns the op executor, which contains the operator computation flow.	-	-	-	-	-

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

The first-phase API performs input parameter validation and returns an error in the following scenarios:

Return Code	Error Code	Description
ACLNN_ERR_PARAM_NULLPTR	161001	The input self, scale, zeroPoint, out, or mask is a null pointer.
ACLNN_ERR_PARAM_INVALID	161002	The data type of self, scale, zeroPoint, out, or mask is not within the supported range.
		The shape of out or mask is inconsistent with that of self.
		The shape of scale or zeroPoint is not 1-dimensional.
		The sizes of scale and zeroPoint are inconsistent.
		The dimension specified by axis is not within the valid range [-self.dim(), self.dim()-1].
		The size of scale or zeroPoint is inconsistent with the size of self on the axis dimension.
		quantMin is greater than quantMax.

aclnnFakeQuantPerChannelAffineCachemask

Parameters:

Parameter	Input/Output	Description
workspace	Input	Memory address of the workspace applied on the Device side.
workspaceSize	Input	Size of the workspace applied on the Device side, obtained from the first-phase API aclnnFakeQuantPerChannelAffineCachemaskGetWorkspaceSize.
executor	Input	Operator executor, which contains the operator computation flow.
stream	Input	Specifies the stream that executes the task.

Return value

aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

Deterministic computation:
- aclnnFakeQuantPerChannelAffineCachemask defaults to a deterministic implementation.
The current new operator FakeQuantPerChannelAffineCachemask does not support float32 and float16 inputs for zero_point. Therefore, this is intercepted inside the aclnn API for now, and the restriction will be lifted once the operator supports it.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_fake_quant_per_channel_affine_cachemask.h"

#define CHECK_RET(cond, return_expr) \
  do {                               \
    if (!(cond)) {                   \
      return_expr;                   \
    }                                \
  } while (0)

#define LOG_PRINT(message, ...)     \
  do {                              \
    printf(message, ##__VA_ARGS__); \
  } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape) {
  int64_t shapeSize = 1;
  for (auto i : shape) {
    shapeSize *= i;
  }
  return shapeSize;
}

int Init(int32_t deviceId, aclrtStream* stream) {
  // Boilerplate: initialize resources.
  auto ret = aclInit(nullptr);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
  ret = aclrtSetDevice(deviceId);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
  ret = aclrtCreateStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
  return 0;
}

template <typename T>
int CreateAclTensor(const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr,
                    aclDataType dataType, aclTensor** tensor) {
  auto size = GetShapeSize(shape) * sizeof(T);
  // Call aclrtMalloc to apply for memory on the device side.
  auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);
  // Call aclrtMemcpy to copy data from the host side to the device memory.
  ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

  // Calculate the strides of the contiguous tensor.
  std::vector<int64_t> strides(shape.size(), 1);
  for (int64_t i = shape.size() - 2; i >= 0; i--) {
    strides[i] = shape[i + 1] * strides[i + 1];
  }

  // Call the aclCreateTensor API to create an aclTensor.
  *tensor = aclCreateTensor(shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND,
                            shape.data(), shape.size(), *deviceAddr);
  return 0;
}

int main() {
  // 1. (Boilerplate) Initialize the device/stream. Refer to the ACL API manual.
  // Fill in the deviceId based on the actual device.
  int32_t deviceId = 0;
  aclrtStream stream;
  auto ret = Init(deviceId, &stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);

  // 2. Construct the input and output. Customize the construction based on the API.
  std::vector<int64_t> selfShape = {1};
  std::vector<int64_t> scaleShape = {1};
  std::vector<int64_t> zeroPointShape = {1};
  std::vector<int64_t> outShape = {1};
  std::vector<int64_t> maskShape = {1};
  void* selfDeviceAddr = nullptr;
  void* scaleDeviceAddr = nullptr;
  void* zeroPointDeviceAddr = nullptr;
  void* outDeviceAddr = nullptr;
  void* maskDeviceAddr = nullptr;
  aclTensor* self = nullptr;
  aclTensor* scale = nullptr;
  aclTensor* zeroPoint = nullptr;
  aclTensor* out = nullptr;
  aclTensor* mask = nullptr;
  std::vector<float> selfHostData{1};
  std::vector<float> scaleHostData{1};
  std::vector<int32_t> zeroPointHostData{1};
  std::vector<float> outHostData{1};
  std::vector<char> maskHostData{1};
  int64_t quantMin = 1;
  int64_t quantMax = 3;
  int axis = 0;
  // Create an aclTensor.
  ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT, &self);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(scaleHostData, scaleShape, &scaleDeviceAddr, aclDataType::ACL_FLOAT, &scale);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(zeroPointHostData, zeroPointShape, &zeroPointDeviceAddr, aclDataType::ACL_INT32, &zeroPoint);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_FLOAT, &out);
  CHECK_RET(ret == ACL_SUCCESS, return ret);
  ret = CreateAclTensor(maskHostData, maskShape, &maskDeviceAddr, aclDataType::ACL_BOOL, &mask);
  CHECK_RET(ret == ACL_SUCCESS, return ret);

  // 3. Call the CANN operator library API. Replace it with the specific API name.
  uint64_t workspaceSize = 0;
  aclOpExecutor* executor;
  // Call the first-phase API of aclnnEye.
  ret = aclnnFakeQuantPerChannelAffineCachemaskGetWorkspaceSize(self, scale, zeroPoint, axis, quantMin, quantMax, out, mask, &workspaceSize, &executor);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnFakeQuantPerChannelAffineCachemaskGetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
  // Apply for device memory based on the workspaceSize calculated by the first-phase API.
  void* workspaceAddr = nullptr;
  if (workspaceSize > 0) {
    ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret);
  }
  // Call the second-phase API of aclnnFakeQuantPerChannelAffineCachemask.
  ret = aclnnFakeQuantPerChannelAffineCachemask(workspaceAddr, workspaceSize, executor, stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnFakeQuantPerChannelAffineCachemask failed. ERROR: %d\n", ret); return ret);

  // 4. (Boilerplate) Synchronously wait for task execution to complete.
  ret = aclrtSynchronizeStream(stream);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);

  // 5. Obtain the output value and copy the result from the device memory to the host memory. Modify based on the specific API definition.
  auto size = GetShapeSize(outShape);
  std::vector<float> resultData(size, 0);
  ret = aclrtMemcpy(resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr,
                    size * sizeof(resultData[0]), ACL_MEMCPY_DEVICE_TO_HOST);
  CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
  for (int64_t i = 0; i < size; i++) {
    LOG_PRINT("result[%ld] is: %f\n", i, resultData[i]);
  }

  // 6. Destroy aclTensor. Modify based on the specific API definition.
  aclDestroyTensor(self);
  aclDestroyTensor(scale);
  aclDestroyTensor(zeroPoint);
  aclDestroyTensor(out);
  aclDestroyTensor(mask);

  // 7. Release device resources. Modify based on the specific API definition.
  aclrtFree(selfDeviceAddr);
  aclrtFree(scaleDeviceAddr);
  aclrtFree(zeroPointDeviceAddr);
  aclrtFree(outDeviceAddr);
  aclrtFree(maskDeviceAddr);
  if (workspaceSize > 0) {
    aclrtFree(workspaceAddr);
  }
  aclrtDestroyStream(stream);
  aclrtResetDevice(deviceId);
  aclFinalize();
  return 0;
}