Documentation
Download
文档中心
aclnnAscendQuantV3

aclnnAscendQuantV3

📄 View Source Code

Applicable Products

ProductSupported
Ascend 950PR/Ascend 950DT√
Atlas A3 training products/Atlas A3 inference products√
Atlas A2 training products/Atlas A2 inference products√
Atlas 200I/500 A2 inference products×
Atlas inference products√
Atlas training products×

Function

  • Description: Performs quantization on the input x. You can set axis to specify the axis corresponding to scale and offset. The shape of scale and offset must be equal to the axis of x specified by axis, or equal to 1.

  • Formula:

    • When sqrtMode is false, the formula is:

      y=round((x∗scale)+offset)y = round((x * scale) + offset)

    • When sqrtMode is true, the formula is:

      y=round((x∗scale∗scale)+offset)y = round((x * scale * scale) + offset)

Function Prototype

Each operator is divided into a two-phase API. You must first call the "aclnnAscendQuantV3GetWorkspaceSize" API to obtain the workspace size required for computation and the executor that encapsulates the operator computation flow, and then call the "aclnnAscendQuantV3" API to perform the computation.

aclnnStatus aclnnAscendQuantV3GetWorkspaceSize(
  const aclTensor *x,
  const aclTensor *scale,
  const aclTensor *offset,
  bool             sqrtMode,
  const char      *roundMode,
  int32_t          dstType,
  int32_t          axis,
  const aclTensor *y,
  uint64_t        *workspaceSize,
  aclOpExecutor  **executor)
aclnnStatus aclnnAscendQuantV3(
  void          *workspace,
  uint64_t       workspaceSize,
  aclOpExecutor *executor,
  aclrtStream    stream)

aclnnAscendQuantV3GetWorkspaceSize

  • Parameters:

    Parameter Input/Output Description Instruction Data Type Data Format Dimension (Shape) Non-Contiguous Tensor
    x (aclTensor*) Input Input to be quantized. Corresponds to `x` in the formula.
    • Nullable tensor.
    • When the data format is ND, if `dstType` is 3, the last dimension of the shape must be divisible by 8; if `dstType` is 29, the last dimension of the shape must be divisible by 2.
    • When the data format is NZ, the shape supports only three dimensions, and the last dimension of the shape must be divisible by 8.
    		 FLOAT32, FLOAT16, BFLOAT16 ND, NZ 1-8 √
    scale (aclTensor*) Input Scale value in quantization. Corresponds to `scale` in the formula.
    • Nullable.
    • `scale` supports a 1D tensor or a multi-dimensional tensor. Its shape is related to the input `x` and the attribute `axis` (when `scale` is a 1D tensor, dimension 0 of `scale` must equal the `axis` dimension of `x` or equal 1; when `scale` is a multi-dimensional tensor, the number of dimensions of `scale` must be the same as that of `x`, the `axis` dimension of `scale` must equal the `axis` dimension of `x` or equal 1, and all other dimensions of `scale` must be 1).
    • The data type and data format must be consistent with those of `x`.
    • When the data format is NZ, all element values of `scale` must be 1.
    		 FLOAT32, FLOAT16, BFLOAT16 ND, NZ 1-8 √
    offset (aclTensor*) Input Optional parameter, the offset value in dequantization. Corresponds to `offset` in the formula.
    • Nullable tensor.
    • The data type and shape must be consistent with `scale`.
    • When the data format is NZ, the value is null, and the data type of offset is consistent with x.
    		 FLOAT32, FLOAT16, BFLOAT16 ND, NZ 1-8 √
    sqrtMode (bool) Input Specifies the logic for scale participation in computation. Corresponds to `sqrtMode` in the formula. When set to true, the formula is y = round((x * scale * scale) + offset); when set to false, the formula is y = round((x * scale) + offset). - - - -
    roundMode (char*) Input Specifies the conversion method for casting to INT8 output.
    • Supported values: round, ceil, trunc, floor, and hybrid.
    • When the data format of input `x` is NZ, only round is supported.
    		 - - - -
    dstType (int32_t) Input Specifies the data type of the output.
    • Supports values 2, 3, 29, 34, 35, and 36, representing INT8, INT32, INT4, HIFLOAT8, FLOAT8_E5M2, and FLOAT8_E4M3FN, respectively.
    • When the data format of input `x` is NZ, supports value 3, representing INT32.
    		 - - - -
    axis (int32_t) Input Specifies the dimension of `x` to which `scale` and `offset` correspond. When the data format of the input `x` is NZ, the value is -1. - - - - -
    y (aclTensor*) Output Quantized computation output. Corresponds to `y` in the formula.
    • Nullable.
    • When the data type is INT32, the last dimension of the shape is 1/8 of the last dimension of `x`, and the other dimensions are the same as `x`; for other data types, the shape is the same as `x`.
    		 INT8, INT32, INT4, HIFLOAT8, FLOAT8_E5M2, FLOAT8_E4M3FN ND, NZ 1-8 √
    workspaceSize (uint64_t*) Output Returns the workspace size to be allocated on the Device side. - - - - -
    executor (aclOpExecutor**) Output Returns the op executor, which contains the operator computation flow. - - - - -

    • Atlas A2 training products/Atlas A2 inference products, Atlas A3 training products/Atlas A3 inference products:

      • When the data format of parameters x, scale, and offset is NZ, the data type only supports FLOAT32.

      • The output parameter y supports only INT8, INT32, and INT4 data types. When the data format is NZ, the data type supports INT32.

      • The input parameter roundMode supports the values round, ceil, trunc, and floor. When the data format of input x is NZ, it supports the value round.

      • The input parameter dstType supports the values 2, 3, and 29, which represent INT8, INT32, and INT4 respectively. When the data format of input x is NZ, it supports the value 3, which represents INT32.

      • The input parameter axis supports specifying the last two dimensions of x (assuming the input x has xDimNum dimensions, the value range of axis is [-2, -1] or [xDimNum-2, xDimNum-1]).

    • Ascend 950PR/Ascend 950DT:

      • The data format of x, scale, and offset does not support NZ.

      • Input parameter roundMode: When dstType is FLOAT8_E5M2 or FLOAT8_E4M3FN, only round is supported. When dstType is HIFLOAT8, round and hybrid are supported. When dstType is another type, round, ceil, trunc, and floor are supported.

      • The input parameter axis supports specifying the last two dimensions of x (assuming the input x has xDimNum dimensions, the value range of axis is [-2, -1] or [xDimNum-2, xDimNum-1]).

    • Atlas inference products:

      • The data type of input parameters x, scale, and offset does not support BFLOAT16, and the data format does not support NZ.

      • The output parameter y only supports the INT8 data type and does not support the NZ data format.

      • The input parameter roundMode supports the values round, ceil, trunc, and floor.

      • The input parameter dstType only supports the value 2, which indicates INT8.

      • The input parameter axis only supports specifying the last dimension of x (assuming the input x has xDimNum dimensions, the value of axis is -1 or xDimNum-1).

  • Return value

    aclnnStatus: return code. For details, see aclnn Return Code.

    The first-phase API performs input parameter validation and returns an error in the following scenarios:

    Return Code Error Code Description
    ACLNN_ERR_PARAM_NULLPTR 161001 The input x, scale, or y is a null pointer.
    ACLNN_ERR_PARAM_INVALID 161002 The data type or data format of x, scale, offset, or y is not within the supported range.
    The shape of x, scale, offset, or y does not meet the constraint requirements.
    The number of dimensions of x is not between 1 and 8.
    roundMode is not within the valid value range.
    dstType is not within the valid value range.
    axis is not within the valid value range.
    When the data type of y is INT4, the size of the last axis of x's shape is not an even number.
    When the data type of y is INT32, the last axis of y's shape is not 1/8 of the size of the last axis of x's shape, or the sizes of the non-last axes of x and y's shapes are inconsistent.

aclnnAscendQuantV3

  • Parameters:

    Parameter Input/Output Description
    workspace Input Memory address of the workspace applied on the Device side.
    workspaceSize Input Size of the workspace applied on the Device side, obtained by the first-phase API aclnnAscendQuantV3GetWorkspaceSize.
    executor Input Operator executor that contains the operator computation flow.
    stream Input Specifies the stream for executing the task.

  • Return value

    aclnnStatus: return code. For details, see aclnn Return Code.

Constraints

  • Deterministic computation:

    • aclnnAscendQuantV3 defaults to a deterministic implementation.

Example

The following provides example code for reference only. For details about compilation and running, see Compile and Run Samples.

#include <iostream>
#include <vector>
#include "acl/acl.h"
#include "aclnnop/aclnn_ascend_quant_v3.h"

#define CHECK_RET(cond, return_expr) \
    do {                             \
        if (!(cond)) {               \
            return_expr;             \
        }                            \
    } while (0)

#define LOG_PRINT(message, ...)         \
    do {                                \
        printf(message, ##__VA_ARGS__); \
    } while (0)

int64_t GetShapeSize(const std::vector<int64_t>& shape)
{
    int64_t shape_size = 1;
    for (auto i : shape) {
        shape_size *= i;
    }
    return shape_size;
}

int Init(int32_t deviceId, aclrtStream* stream)
{
    // Boilerplate for resource initialization.
    auto ret = aclInit(nullptr);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclInit failed. ERROR: %d\n", ret); return ret);
    ret = aclrtSetDevice(deviceId);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSetDevice failed. ERROR: %d\n", ret); return ret);
    ret = aclrtCreateStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtCreateStream failed. ERROR: %d\n", ret); return ret);
    return 0;
}

template <typename T>
int CreateAclTensor(
    const std::vector<T>& hostData, const std::vector<int64_t>& shape, void** deviceAddr, aclDataType dataType,
    aclTensor** tensor)
{
    auto size = GetShapeSize(shape) * sizeof(T);
    // Call aclrtMalloc to allocate device-side memory.
    auto ret = aclrtMalloc(deviceAddr, size, ACL_MEM_MALLOC_HUGE_FIRST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMalloc failed. ERROR: %d\n", ret); return ret);

    // Copy data from the host to the device using aclrtMemcpy.
    ret = aclrtMemcpy(*deviceAddr, size, hostData.data(), size, ACL_MEMCPY_HOST_TO_DEVICE);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtMemcpy failed. ERROR: %d\n", ret); return ret);

    // Calculate the strides of a contiguous tensor.
    std::vector<int64_t> strides(shape.size(), 1);
    for (int64_t i = shape.size() - 2; i >= 0; i--) {
        strides[i] = shape[i + 1] * strides[i + 1];
    }

    // Call the aclCreateTensor API to create an aclTensor.
    *tensor = aclCreateTensor(
        shape.data(), shape.size(), dataType, strides.data(), 0, aclFormat::ACL_FORMAT_ND, shape.data(), shape.size(),
        *deviceAddr);
    return 0;
}

int main()
{
    // 1. (Boilerplate) Initialize the device/stream. See the ACL API manual.
    // Set deviceId based on the actual device.
    int32_t deviceId = 0;
    aclrtStream stream;
    auto ret = Init(deviceId, &stream);
    // Handle the check based on your requirements.
    CHECK_RET(ret == 0, LOG_PRINT("Init acl failed. ERROR: %d\n", ret); return ret);
    // 2. Construct the input and output. You need to customize the construction based on the API.
    std::vector<int64_t> selfShape = {4, 2};
    std::vector<int64_t> scaleShape = {2};
    std::vector<int64_t> offsetShape = {2};
    std::vector<int64_t> outShape = {4, 2};
    void* selfDeviceAddr = nullptr;
    void* scaleDeviceAddr = nullptr;
    void* offsetDeviceAddr = nullptr;
    void* outDeviceAddr = nullptr;
    aclTensor* self = nullptr;
    aclTensor* scale = nullptr;
    aclTensor* offset = nullptr;
    aclTensor* out = nullptr;
    std::vector<float> selfHostData = {0, 1, 2, 3, 4, 5, 6, 7};
    std::vector<float> scaleHostData = {1, 2};
    std::vector<float> offsetHostData = {1, 2};
    std::vector<int8_t> outHostData = {0, 0, 0, 0, 0, 0, 0, 0};
    // Create the self aclTensor.
    ret = CreateAclTensor(selfHostData, selfShape, &selfDeviceAddr, aclDataType::ACL_FLOAT, &self);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the scale aclTensor.
    ret = CreateAclTensor(scaleHostData, scaleShape, &scaleDeviceAddr, aclDataType::ACL_FLOAT, &scale);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the offset aclTensor.
    ret = CreateAclTensor(offsetHostData, offsetShape, &offsetDeviceAddr, aclDataType::ACL_FLOAT, &offset);
    CHECK_RET(ret == ACL_SUCCESS, return ret);
    // Create the out aclTensor.
    ret = CreateAclTensor(outHostData, outShape, &outDeviceAddr, aclDataType::ACL_INT8, &out);
    CHECK_RET(ret == ACL_SUCCESS, return ret);

    const int32_t dstType = 2;
    const int32_t axis = -1;
    bool sqrtMode = false;
    const char* roundMode = "round";

    // 3. Call the CANN operator library API. Replace with the actual API name.
    uint64_t workspaceSize = 0;
    aclOpExecutor* executor;
    // Call the first-phase API of aclnnAscendQuantV3.
    ret = aclnnAscendQuantV3GetWorkspaceSize(
        self, scale, offset, sqrtMode, roundMode, dstType, axis, out, &workspaceSize, &executor);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAscendQuantV3GetWorkspaceSize failed. ERROR: %d\n", ret); return ret);
    // Apply for device memory based on the workspaceSize calculated by the first-phase API.
    void* workspaceAddr = nullptr;
    if (workspaceSize > 0) {
        ret = aclrtMalloc(&workspaceAddr, workspaceSize, ACL_MEM_MALLOC_HUGE_FIRST);
        CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("allocate workspace failed. ERROR: %d\n", ret); return ret;);
    }
    // Call the second-phase API of aclnnAscendQuantV3.
    ret = aclnnAscendQuantV3(workspaceAddr, workspaceSize, executor, stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclnnAscendQuantV3 failed. ERROR: %d\n", ret); return ret);
    // 4. (Boilerplate) Synchronously wait for the task execution to complete.
    ret = aclrtSynchronizeStream(stream);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("aclrtSynchronizeStream failed. ERROR: %d\n", ret); return ret);
    // 5. Obtain the output value. Copy the result from the device memory to the host. Modify based on the specific API definition. Check the data in resultData.
    auto size = GetShapeSize(outShape);
    std::vector<int8_t> resultData(size, 0);
    ret = aclrtMemcpy(
        resultData.data(), resultData.size() * sizeof(resultData[0]), outDeviceAddr, size * sizeof(int8_t),
        ACL_MEMCPY_DEVICE_TO_HOST);
    CHECK_RET(ret == ACL_SUCCESS, LOG_PRINT("copy result from device to host failed. ERROR: %d\n", ret); return ret);
    for (int64_t i = 0; i < size; i++) {
        LOG_PRINT("result[%ld] is: %d\n", i, resultData[i]);
    }

    // 6. Release aclTensor. Modify based on the specific API definition.
    aclDestroyTensor(self);
    aclDestroyTensor(scale);
    aclDestroyTensor(offset);
    aclDestroyTensor(out);

    // 7. Release device resources. Modify based on the specific API definition.
    aclrtFree(selfDeviceAddr);
    aclrtFree(scaleDeviceAddr);
    aclrtFree(offsetDeviceAddr);
    aclrtFree(outDeviceAddr);
    if (workspaceSize > 0) {
        aclrtFree(workspaceAddr);
    }
    aclrtDestroyStream(stream);
    aclrtResetDevice(deviceId);
    aclFinalize();
    return 0;
}
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details