SetDeqScale

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Sets the value of the DEQSCALE register.

Prototype

For s322f16 scenarios of AddDeqRelu/Cast/CastDeq:

__aicore__ inline void SetDeqScale(half scale)

Used in the CastDeq (isVecDeq=false) scenario

__aicore__ inline void SetDeqScale(float scale, int16_t offset, bool signMode)

Used in the CastDeq (isVecDeq=true) scenario

template <typename T>
__aicore__ inline void SetDeqScale(const LocalTensor<T>& vdeq, const VdeqInfo& vdeqInfo)

Parameters

**Table 1** Parameters in the template
Parameter	Description
T	Data type of the input quantization tensor. The supported data type is uint64_t.

Table 2 Parameters

Parameter

Input/Output

Description

scale (half)

Input

Scale quantization parameter, which is of the half type.

Atlas 350 Accelerator Card: used in s322f16 scenarios of AddDeqRelu/CastDeq/Cast.

Atlas A3 training product/Atlas A3 inference product: used in s322f16 scenarios of AddDeqRelu/Cast/CastDeq.

Atlas A2 training product/Atlas A2 inference product: used in s322f16 scenarios of AddDeqRelu/Cast/CastDeq.

Atlas inference product AI Core: used in s322f16 scenarios of AddDeqRelu or Cast.

scale (float)

Input

Scale quantization parameter, which is of the float type.

Used by the CastDeq (isVecDeq=false) instruction to set the value of the DEQSCALE register.

offset

Input

Offset quantization parameter, which is of the int16_t type. Only the first nine bits are valid.

Used in the CastDeq (isVecDeq=false) scenario.

signMode

Input

Whether the quantization result contains a sign. The value is of the Boolean type.

Used in the CastDeq (isVecDeq=false) scenario.

vdeq

Input

Used in the CastDeq (isVecDeq=true) scenario. A 128-byte quantized tensor is input.

The type is LocalTensor, and TPosition can be VECIN, VECCALC, or VECOUT.

The start address of LocalTensor must be 32-byte aligned.

vdeqInfo

Input

Data structure for storing the quantized tensor information. The structure contains 16 groups of quantization parameters of the quantized tensor.

const uint8_t VDEQ_TENSOR_SIZE = 16;

struct VdeqInfo {
    __aicore__ VdeqInfo() {}
    __aicore__ VdeqInfo(const float vdeqScaleIn[VDEQ_TENSOR_SIZE], const int16_t vdeqOffsetIn[VDEQ_TENSOR_SIZE],
        const bool vdeqSignModeIn[VDEQ_TENSOR_SIZE])
    {
        for (int32_t i = 0; i < VDEQ_TENSOR_SIZE; ++i) {
            vdeqScale[i] = vdeqScaleIn[i];
            vdeqOffset[i] = vdeqOffsetIn[i];
            vdeqSignMode[i] = vdeqSignModeIn[i];
        }
    }

    float vdeqScale[VDEQ_TENSOR_SIZE] = { 0 };
    int16_t vdeqOffset[VDEQ_TENSOR_SIZE] = { 0 };
    bool vdeqSignMode[VDEQ_TENSOR_SIZE] = { 0 };
};

vdeqScale: array of the float type, used to store the scale parameter scale₀ - scale₁₅ of the quantized tensor.
vdeqOffset: array of the int16_t type, used to store the offset parameter offset₀ - offset₁₅ of the quantized tensor.
vdeqSignMode: array of the Boolean type, used to store the signMode parameter signMode₀ - signMode₁₅ of the quantized tensor.

Returns

None

Restrictions

None

Examples

SetDeqScale(half scale)

// Used together with the s322f16 scenario of Cast.
// dstLocal is a LocalTensor of the half type, and srcLocal is a LocalTensor of the int32_t type.
uint32_t srcSize = 256; // Number of elements involved in computation
half scale = 1.0; // The quantization parameter is 1.
AscendC::SetDeqScale(scale);
// dst = src
AscendC::Cast(dstLocal, srcLocal, AscendC::RoundMode::CAST_NONE, srcSize);

Result example:

Input (srcLocal):
[1, 2, 3, 4, 5, 6, ... 256]
Output (dstLocal):
[1, 2, 3, 4, 5, 6, ... 256]

SetDeqScale(float scale, int16_t offset, bool signMode)

// Used together with CastDeq (isVecDeq=false).
// dstLocal is a LocalTensor of the int8_t type, and srcLocal is a LocalTensor of the int16_t type.
uint32_t srcSize = 256; // Number of elements involved in computation
float scale = 1.0; // The quantization parameter is 1.
int16_t offset = 0; // No offset
bool signMode = true; // dstLocal is of the int8_t type and is a signed number.
AscendC::SetDeqScale(scale, offset, signMode);
// dst = src
AscendC::CastDeq<int8_t, int16_t, false, false>(dstLocal, srcLocal, srcSize);

Result example:

Input (srcLocal):
[[  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15 ]
 [ 16  17 18  19  20  21  22  23  24  25  26  27  28  29  30  31  ]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47 ]
 [ 48  49 50  51  52  53  54  55  56  57  58  59  60  61  62  63  ]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79 ]
 [ 80  81 82  83  84  85  86  87  88  89  90  91  92  93  94  95  ]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111 ]
 [ 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 ]
Output (dstLocal):
// Write the upper half block of dstLocal.
[[0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]
[112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0]]

SetDeqScale(const LocalTensor<T>& vdeq, const VdeqInfo& vdeqInfo)

// Used together with CastDeq (isVecDeq=true).
// dstLocal is a LocalTensor of the int8_t type, and srcLocal is a LocalTensor of the int16_t type.
uint32_t srcSize = 256; // Number of elements involved in computation
float vdeqScale[16] = { 0 };
int16_t vdeqOffset[16] = { 0 };
bool vdeqSignMode[16] = { 0 };
for (int i = 0; i < 16; i++) {
    vdeqScale[i] = 1.0; // The quantization parameter is 1.
    vdeqOffset[i] = 0; // No offset
    vdeqSignMode[i] = true; // dstLocal is of the int8_t type and is a signed number.
}
AscendC::VdeqInfo vdeqInfo(vdeqScale, vdeqOffset, vdeqSignMode);
AscendC::SetDeqScale<uint64_t>(tmpBuffer, vdeqInfo);
// dst = src
AscendC::CastDeq<int8_t, int16_t, true, true>(dstLocal, srcLocal, srcSize);

Result example:

Input (srcLocal):
[[  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15 ]
 [ 16  17 18  19  20  21  22  23  24  25  26  27  28  29  30  31  ]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47 ]
 [ 48  49 50  51  52  53  54  55  56  57  58  59  60  61  62  63  ]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79 ]
 [ 80  81 82  83  84  85  86  87  88  89  90  91  92  93  94  95  ]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111 ]
 [ 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 ]
Output (dstLocal):
// Write the lower half block of dstLocal.
[[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127]]

Parent topic: Quantization Settings