Documentation
Download

Fixpipe

Applicability

Product

Supported

Atlas 350 Accelerator Card

Only APIs that contain FixpipeParamsV220 or FixpipeParamsArch3510 are supported.

Atlas A3 training product / Atlas A3 inference product

Only APIs that contain FixpipeParamsV220 are supported.

Atlas A2 training product / Atlas A2 inference product

Only APIs that contain FixpipeParamsV220 are supported.

Atlas 200I/500 A2 inference product

Only APIs that contain FixpipeParamsM300 are supported.

Atlas inference product AI Core

x

Atlas inference product Vector Core

x

Atlas training product

x

Function Usage

Processes the result after the matrix computation is complete. For example, the computation result is quantized, and the data is moved from CO1 to the global memory.

In the preceding figure, the data transfer path Fixpipe Buffer -> UB -> L1 Buffer is supported only in NPU_ARCH 351x.

The memory layout of FixPipe Buffer changes depending on which quantization mode is being used.

Prototype

  • Pass FixpipeParamsV220.
    • Path CO1 -> GM, with tensor quantization disabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const GlobalTensor<T>& dst, const LocalTensor<U>& src, const FixpipeParamsV220& intriParams)
    • Path CO1 -> GM, with tensor quantization enabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR, typename S = uint64_t, typename Std::enable_if<Std::is_same<PrimT<S>, uint64_t>::value, bool>::type = true>
__aicore__ inline void Fixpipe(const GlobalTensor<T>& dst, const LocalTensor<U>& src, const LocalTensor<S>& cbufWorkspace, const FixpipeParamsV220& intriParams)
  • Pass FixpipeParamsM300.
    • Path CO1 -> UB, with tensor quantization disabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const LocalTensor<T>& dst, const LocalTensor<U>& src, const FixpipeParamsM300& intriParams)
    • Path CO1 -> UB, with tensor quantization enabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR, typename S = uint64_t, typename Std::enable_if<Std::is_same<PrimT<S>, uint64_t>::value, bool>::type = true>
__aicore__ inline void Fixpipe(const LocalTensor<T>& dst, const LocalTensor<U>& src, const LocalTensor<S>& cbufWorkspace, const FixpipeParamsM300& intriParams)
    • Path CO1 -> GM, with tensor quantization disabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const GlobalTensor<T>& dst, const LocalTensor<U>& src, const FixpipeParamsM300& intriParams)
    • Path CO1 -> GM, with tensor quantization enabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR, typename S = uint64_t, typename Std::enable_if<Std::is_same<PrimT<S>, uint64_t>::value, bool>::type = true>
__aicore__ inline void Fixpipe(const GlobalTensor<T>& dst, const LocalTensor<U>& src, const LocalTensor<S>& cbufWorkspace, const FixpipeParamsM300& intriParams)
  • Pass FixpipeParamsArch3510.
    • CO1 -> L1 Buffer, with tensor quantization disabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const LocalTensor<T>& dst, const LocalTensor<U>& src, const FixpipeParamsArch3510<config.format>& intriParams)
    • CO1 -> L1 Buffer, with tensor quantization enabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const LocalTensor<T>& dst, const LocalTensor<U>& src, const LocalTensor<uint64_t>& cbufWorkspace, const FixpipeParamsArch3510<config.format>& intriParams)
    • Path CO1 -> UB, with tensor quantization disabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const LocalTensor<T>& dst, const LocalTensor<U>& src, const FixpipeParamsArch3510<config.format>& intriParams)
    • Path CO1 -> UB, with tensor quantization enabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const LocalTensor<T>& dst, const LocalTensor<U>& src, const LocalTensor<uint64_t>& cbufWorkspace, const FixpipeParamsArch3510<config.format>& intriParams)
    • Path CO1 -> GM, with tensor quantization disabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const GlobalTensor<T>& dst, const LocalTensor<U>& src, const FixpipeParamsArch3510<config.format>& intriParams)
    • Path CO1 -> GM, with tensor quantization enabled: 
      1
2
      		 
      template <typename T, typename U, const FixpipeConfig& config = CFG_ROW_MAJOR>
__aicore__ inline void Fixpipe(const GlobalTensor<T>& dst, const LocalTensor<U>& src, const LocalTensor<uint64_t>& cbufWorkspace, const FixpipeParamsArch3510<config.format>& intriParams)

Parameters

Table 1 Template parameters

Parameter

Description

T

Data type of the destination operand.

U

Data type of the source operand.

config

Fixpipe configuration parameter. The type is FixpipeConfig. The values are as follows:

  • CFG_ROW_MAJOR (default value): NZ2ND is enabled, and the output data format is ND.
  • CFG_NZ: NZ2ND is disabled. The output data format is NZ.
  • CFG_COLUMN_MAJOR: NZ2DN is enabled. The output data format is DN.
 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
 
struct FixpipeConfig {
    CO2Layout format;
    bool isToUB; // Specifies whether the destination address is in the UB.
};
enum class CO2Layout : uint8_t {
    NZ = 0, // The output data format is NZ.
    ROW_MAJOR, // Enable NZ2ND. The output data format is ND.
    COLUMN_MAJOR, // NZ2DN is enabled, and the output data format is DN.
};
constexpr FixpipeConfig CFG_NZ = {CO2Layout::NZ};
constexpr FixpipeConfig CFG_ROW_MAJOR = {CO2Layout::ROW_MAJOR};
constexpr FixpipeConfig CFG_COLUMN_MAJOR = {CO2Layout::COLUMN_MAJOR};

S

Data type of cbufWorkspace.

  • When the destination operand, source operand, and cbufWorkspace are of the basic data type, the template parameter S must be of type uint64_t. Otherwise, the compilation fails.
  • When the destination operand, source operand, and cbufWorkspace are of the TensorTrait type, the LiteType of the template parameter S must be of type uint64_t. Otherwise, the compilation fails.

The template parameter S is used only for checking the preceding data types.
Table 2 Parameters

Parameter

Input/Output

Description

dst

Output

Destination operand, which is of the LocalTensor or GlobalTensor type.

  • For the LocalTensor type:

    For the Atlas 200I/500 A2 inference product , the supported data types are int8_t, uint8_t, half, bfloat16_t, float, half, and int32_t.

    For the Atlas 350 Accelerator Card, the supported data types are int8_t, uint8_t, half, bfloat16_t, float, half, and int32_t.

  • For the GlobalTensor type:

    For the Atlas 350 Accelerator Card, the supported data types are int8_t, uint8_t, hifloat8_t, fp8_e4m3fn_t, half, bfloat16_t, int32_t, and float.

    For the Atlas A3 training product / Atlas A3 inference product , the supported data types are int8_t, uint8_t, half, bfloat16_t, int32_t, and float.

    For the Atlas A2 training product / Atlas A2 inference product , the supported data types are int8_t, uint8_t, half, bfloat16_t, int32_t, and float.

    For the Atlas 200I/500 A2 inference product , the supported data types are int8_t, uint8_t, half, bfloat16_t, int32_t, and float.

    The data format is NZ, DN, or ND. After Fixpipe processing, extra data allocated for matrix computation will be deleted upon completion of quantization.

src

Input

Source operand. The supported TPosition is CO1, which outputs the computation result of the MMAD API. For details about the definition of the LocalTensor data structure, see LocalTensor. The supported data types are float and int32_t, the supported TPosition is CO1, and the data format is NZ. The start address must be 64-byte aligned.

intriParams

Input

Fixpipe transfer parameters. For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_fixpipe.h. Replace ${INSTALL_DIR} with the actual path for storing files after the CANN software is installed.

For details about the parameter description, see Table 3.

cbufWorkspace

Input

Quantization parameter, which is of the LocalTensor<uint64_t> type. The supported TPosition is A1. This parameter is supported only when quantPre is set to VDEQF16, VQF322B8_PRE, or VREQ8. For details about quantPre, see quantPre in FixpipeParamsV220 or FixpipeParamsM300 structure.
Table 3 FixPipe data transfer parameters in FixpipeParamsV220 or FixpipeParamsM300

Parameter

Data Type

Description

nSize

Input

Size of the source NZ matrix in the N direction.

  • NZ2ND disabled

    If channelSplit is enabled, the value of nSize must be a multiple of 8. Value range: nSize ∈ [1, 4095].

    If channelSplit is disabled, the value of nSize must be a multiple of 16. Value range: nSize ∈ [1, 4095].

  • NZ2ND enabled

    Value range: nSize ∈ [1, 4095].

mSize

Input

Size of the source NZ matrix in the M direction.

  • NZ2ND disabled

    Value range: mSize ∈ [1, 65535].

  • NZ2ND enabled

    Value range: mSize ∈ [1, 8192].

srcStride

Input

Stride between the start addresses of adjacent Z rows in the source NZ matrix. Value range: srcStride ∈ [0, 65535]. Unit: C0_Size (16 × sizeof(T), where T is the data type of src).

dstStride

Input
  • NZ2ND disabled

    Stride between the start addresses of adjacent Z rows in the destination NZ matrix. The value cannot be 0. Unit: data block (32 bytes).

  • NZ2ND enabled

    Number of elements in each row of the destination ND matrix. The value cannot be 0. Unit: element.

quantPre

Input

QuantMode_t is an enumeration type used to control the quantization mode. The default value is QuantMode_t::NoQuant, indicating that the quantization function is disabled. The options of QuantMode_t are as follows:

  • NoQuant: No quantization is applied.
  • F322F16: float32 is cast to float16, with the CAST_RINT mode used.
  • F322BF16: float32 is cast to bfloat16_t, with the CAST_RINT mode used.
  • DEQF16: int32_t is quantized to float16 via scalar quantization. The output does not support the INF_NAN mode.
  • VDEQF16: int32_t is quantized to float16 via tensor quantization. The output does not support the INF_NAN mode.
  • QF322B8_PRE: float32 is quantized to uint8_t or int8_t via scalar quantization.
  • VQF322B8_PRE: float32 quantized to uint8_t or int8_t via tensor quantization.
  • REQ8: int32_t is quantized to uint8_t or int8_t via scalar quantization.
  • VREQ8: int32_t is quantized to uint8_t or int8_t via tensor quantization.

deqScalar

Input

Scalar quantization parameter, which indicates a single scale value. This parameter needs to be set when quantPre is set to scalar quantization. The supported data type is uint64_t.

ndNum

Input

Number of source NZ matrices, that is, the number of ND matrices to be transferred. Value range: ndNum ∈ [1, 65535].

srcNdStride

Input

Stride between the start addresses of different NZ matrices. Value range: srcNdStride ∈ [1, 512] (unit: data block (16 × C0_Size)). When ndNum is set to 1, srcNdStride is set to 0 and does not take effect.

dstNdStride

Input

Stride between the start addresses of the destination adjacent ND matrices. Value range: dstNdstride ∈ [1, 65535] (unit: element). When ndNum is set to 1, dstNdStride is set to 0 and does not take effect.

reluEn

Input

ReLU switch. false: The ReLU function is disabled. true: The ReLU function is enabled.

unitFlag

Input

Fine-grained parallelism between Mmad and Fixpipe instructions. After this function is enabled, the computation result is moved out each time the hardware completes a fractal computation. This function is not applicable to scenarios where accumulation is performed in the L0C buffer. The options are as follows:

0: Value reserved.

2: unitFlag enabled. After hardware executes the instruction, the register is not configured.

3: unitFlag enabled. After hardware executes the instruction, unitFlag is disabled.

To enable this function, set unitFlag of the Fixpipe instruction to 3.

isChannelSplit

Input

Whether to enable ChannelSplit. The default value is false, indicating that this function is disabled. ChannelSplit can be enabled only when src and dst are both float. In addition, ChannelSplit and NZ2ND cannot be enabled at the same time.
Table 4 Parameters in the FixpipeParamsArch3510 structure

Parameter

Data Type

Description

nSize

Input

Size of the source NZ matrix in the N direction.

  • NZ2ND disabled

    If channelSplit is enabled, the value of nSize must be a multiple of 8. Value range: nSize ∈ [1, 4095].

    If channelSplit is disabled, the value of nSize must be a multiple of 16. Value range: nSize ∈ [1, 4095].

  • NZ2ND enabled

    Value range: nSize ∈ [1, 4095].

    • For the CO1 -> UB/L1 path, nSize × sizeof(T) must be a multiple of 32.

mSize

Input

Size of the source NZ matrix in the M direction.

Value range: mSize ∈ [1, 65535].

  • For the CO1 -> UB/L1 path with NZ2DN enabled, mSize × sizeof(T) must be a multiple of 32.

srcStride

Input

Stride between the start addresses of adjacent Z rows in the source NZ matrix. Value range: srcStride ∈ [0, 65535]. Unit: C0_Size (16 × sizeof(T), where T is the data type of src).

dstStride

Input
  • NZ2ND disabled

    Stride between the start addresses of adjacent Z rows in the destination NZ matrix. The value cannot be 0. Unit: element.

  • NZ2ND/NZ2DN enabled

    Number of elements in each row of the destination ND matrix. The value cannot be 0. Unit: element.

quantPre

Input

QuantMode_t is an enumeration type used to control the quantization mode. The default value is QuantMode_t::NoQuant, indicating that the quantization function is disabled. The options of QuantMode_t are as follows:

  • NoQuant: No quantization is applied.
  • F322F16: float32 is cast to float16, with the CAST_RINT mode used.
  • F322BF16: float32 is cast to bfloat16_t, with the CAST_RINT mode used.
  • DEQF16: int32_t is quantized to float16 via scalar quantization. The output does not support the INF_NAN mode.
  • VDEQF16: int32_t is quantized to float16 via tensor quantization. The output does not support the INF_NAN mode.
  • QF322B8_PRE: float32 is quantized to uint8_t or int8_t via scalar quantization.
  • VQF322B8_PRE: float32 quantized to uint8_t or int8_t via tensor quantization.
  • REQ8: int32_t is quantized to uint8_t or int8_t via scalar quantization.
  • VREQ8: int32_t is quantized to uint8_t or int8_t via tensor quantization.
  • QF322FP8_PRE: float32 is quantized to fp8_e4m3fn_t via scalar quantization.
  • VQF322FP8_PRE: float32 is quantized to fp8_e4m3fn_t via tensor quantization.
  • QF322HIF8_PRE: float32 is quantized to hifloat8_t (Half to Away Round) via scalar quantization.
  • VQF322HIF8_PRE: float32 is quantized to hifloat8_t (Half to Away Round) via tensor quantization.
  • QF322HIF8_PRE_HYBRID: float32 is quantized to hifloat8_t (Hybrid Round) via scalar quantization.
  • VQF322HIF8_PRE_HYBRID: float32 is quantized to hifloat8_t (Hybrid Round) via tensor quantization.
  • QS322BF16_PRE: int32_t is quantized to bfloat16_t via scalar quantization.
  • VQS322BF16_PRE: int32_t is quantized to bfloat16_t via tensor quantization.
  • QF322F16_PRE: float32 is quantized to float16 via scalar quantization.
  • VQF322F16_PRE: float32 is quantized to float16 via tensor quantization.
  • QF322BF16_PRE: float32 is quantized to bfloat16_t via scalar quantization.
  • VQF322BF16_PRE: float32 is quantized to bfloat16_t via tensor quantization.
  • QF322F32_PRE: float32 is quantized back into float32 via scalar quantization. The precision of this quantization mode cannot reach dual-0.01%, but can reach dual-0.1%. If the precision is required to be dual-0.01%, you are advised to use the AscendDeQuant high-level API.
  • VQF322F32_PRE: float32 is quantized back into float32 via tensor quantization. The precision of this quantization mode cannot reach dual-0.01%, but can reach dual-0.1%. If the precision is required to be dual-0.01%, you are advised to use the AscendDeQuant high-level API.

deqScalar

Input

Scalar quantization parameter, which indicates a single scale value. This parameter needs to be set when quantPre is set to scalar quantization. The supported data type is uint64_t.

reluEn

Input

ReLU switch. false: The ReLU function is disabled. true: The ReLU function is enabled.

unitFlag

Input

Fine-grained parallelism between Mmad and Fixpipe instructions. After this function is enabled, the computation result is moved out each time the hardware completes a fractal computation. This function is not applicable to scenarios where accumulation is performed in the L0C buffer. The options are as follows:

0: Value reserved.

2: unitFlag enabled. After hardware executes the instruction, the register is not configured.

3: unitFlag enabled. After hardware executes the instruction, unitFlag is disabled.

To enable this function, set unitFlag of the Fixpipe instruction to 3.

TransformParams

Input

Template parameter-based type selector. It automatically selects the corresponding parameter type at compile time according to different CO2Layouts.

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
 
template <CO2Layout format>
struct TransformParams {};
template <>
struct TransformParams<CO2Layout::NZ> {
    __aicore__ inline TransformParams(){};
    using PARAMS = uint8_t;
};
template <>
struct TransformParams<CO2Layout::ROW_MAJOR> {
    __aicore__ inline TransformParams(){};
    using PARAMS = Nz2NdParams;
};
template <>
struct TransformParams<CO2Layout::COLUMN_MAJOR> {
    __aicore__ inline TransformParams(){};
    using PARAMS = Nz2DnParams;
};

The CO2Layout types are as follows:

  • ROW_MAJOR

    When this mode is enabled, the instruction performs data transfer from L0C to the destination with NZ2ND conversion applied.

    1
2
3
4
5
    		 
    struct Nz2NdParams {
    uint16_t ndNum = 1;
    uint16_t srcNdStride = 0;
    uint32_t dstNdStride = 0; 
};

    ndNum: number of source NZ matrices, that is, the number of ND matrices to be transferred. Value range: ndNum ∈ [1, 65535].

    srcNdStride: stride between the start addresses of different NZ matrices. Value range: srcNdStride ∈ [0, 65535], in the unit of C0_SIZE. When ndNum is set to 1, srcNdStride is set to 0 and does not take effect.

    dstNdStride: stride between the start addresses of the adjacent destination ND matrices. Value range: dstNdstride ∈ [1, 232 – 1] (unit: element). When ndNum is set to 1, dstNdStride is set to 0 and does not take effect.

  • COLUMN_MAJOR

    When this mode is enabled, the instruction performs data transfer from L0C to the destination with NZ2DN conversion applied.

    1
2
3
4
5
6
    		 
    struct Nz2DnParams {
    uint16_t dnNum = 1;
    uint16_t srcNzMatrixStride = 0;
    uint32_t dstDnMatrixStride = 0;
    uint16_t srcNzC0Stride = 0; 
};

    dnNum: number of DN matrices to be transmitted. Value range: dnNum ∈ [1, 65535].

    srcNzMatrixStride: head-to-head stride between different source NZ matrices. The unit is C0_SIZE.

    dstDnMatrixStride: stride between the start addresses of the adjacent destination DN matrices. Value range: dstDnMatrixStride ∈ [1, 232 – 1] (unit: element).

    srcNzC0Stride: head-to-head stride between the addresses of adjacent rows in the NZ tile of the source matrix. The unit is C0_SIZE.

  • NZ

    When this mode is enabled, it works as a standard DMA mode for regular data transfer from L0C to the destination.

dualDstCtrl

Input

Dual-destination mode control. When the dual-destination mode is enabled, the M × N matrix in L0C is split into two halves and written to the UB of two sub-blocks simultaneously. The first half is written to SUB BLOCK0, and the second half to SUB BLOCK1.

2'b00: single-destination mode. The entire matrix is written to the target UB configured by using the subBlockId parameter.

2'b01: dual-destination mode. The matrix is split by M. Each UB receives data of size M/2 × N. M must be a multiple of 2.

2'b10: dual-destination mode. The matrix is split by N. Each UB receives data of size M × N/2. M must be a multiple of 32.

2'b11: reserved.

dualDstCtrl is only supported in standard transfer mode and NZ2ND transfer scenarios. It does not support the real-time function scenarios.

subBlockId

Input

Number of the target UB when the single-destination mode is enabled.

isChannelSplit

Input

Whether to enable ChannelSplit. The default value is false, indicating that this function is disabled. ChannelSplit can be enabled only when src and dst are both float. In addition, ChannelSplit and NZ2ND cannot be enabled at the same time.

The following is an example of parameter settings (using the Fixpipe API to transfer and remove dummy data) and description when NZ2ND is disabled.

If the number of data elements along the M dimension is not a multiple of 16, extra dummy data will be read during transfer and discarded after writing to the destination. A matrix block is defined as a contiguous 16 × 16 data block. The number of data blocks is calculated as the ceiling of M/16. The length of a matrix block is M × 16 × sizeof(T), where T is the data type.

Single transfer mode:

  • nSize = 48 indicates that the to-be-transferred matrix (blue area in the figure) in the source NZ matrix has 48 elements along the N dimension.
  • mSize = 24 indicates that the to-be-transferred matrix in the source NZ matrix has 24 elements along the M dimension.
  • srcStride = 64 indicates the stride between the start addresses of adjacent Z-tiles of the to-be-transferred matrix in the source NZ matrix. The interval between the start addresses of the first and the second blue Z-tiles is 64 × C0_Size.
  • dstStride = 40 indicates the stride between the start addresses of adjacent Z-tiles in the destination NZ matrix. The interval between the start addresses of the first and the second blue Z-tiles is 40 × 32 bytes.
Figure 1 Single transfer mode settings (NZ2ND disabled)

Dual-destination control mode:

As shown in the figure below, dual-destination mode is enabled under standard transfer mode. Data can be split along the M or N dimension. During splitting along the M dimension, M must be a multiple of 2. During splitting along the N dimension, N must be a multiple of 2.

Splitting along the N dimension:

  • nSize = 32 indicates that the to-be-transferred matrix in the source NZ matrix has 32 elements along the N dimension.
  • mSize = 48 indicates that the to-be-transferred matrix in the source NZ matrix has 48 elements along the M dimension.
  • srcStride = 64 indicates the stride between the start addresses of adjacent Z-tiles of the to-be-transferred matrix in the source NZ matrix. The interval between the start addresses of the first and the second Z-tiles is 64 × C0_Size.
  • dstStride = 64 indicates the stride between the start addresses of adjacent Z-tiles in the destination NZ matrix. The interval between the start addresses of the Z-tiles in UB0 and UB1 is 64 × C0_Size.

Splitting along the M dimension:

  • nSize = 32 indicates that the to-be-transferred matrix in the source NZ matrix has 32 elements along the N dimension.
  • mSize = 24 indicates that the to-be-transferred matrix in the source NZ matrix has 24 elements along the M dimension.
  • srcStride = 64 indicates the stride between the start addresses of adjacent Z-tiles of the to-be-transferred matrix in the source NZ matrix. The interval between the start addresses of the first and the second Z-tiles is 64 × C0_Size.
  • dstStride = 40 indicates the stride between the start addresses of adjacent Z-tiles in the destination NZ matrix. The interval between the start addresses of the first and the second Z-tiles in UB0 (or UB1) is 40 × C0_Size.
Figure 2 Dual-destination transfer mode settings (both NZ2ND and NZ2DN disabled)

If NZ2ND is enabled, an example of parameter settings and the description are as follows:

  • ndNum = 2 indicates that the number of source NZ matrices is 2. In the figure, the blue area is NZ matrix 1 and the purple area is NZ matrix 2.
  • nSize = 32 indicates that the source NZ matrix (blue area in the figure) has 32 elements along the N dimension.
  • mSize = 48 indicates that the source NZ matrix has 48 elements along the M dimension.
  • srcStride = 64 indicates the stride between the start addresses of adjacent Z-tiles in the source NZ matrix. The interval between the start addresses of the first and the second blue Z-tiles is 64 × C0_Size.
  • dstStride = 64 indicates that each row in the destination ND matrix contains 64 elements.
  • srcNdStride = 16 indicates that the stride between the start addresses of different NZ matrices is 16 × 16 × C0_Size.
  • dstNdStride = 4096 indicates that the stride between the start addresses of adjacent destination ND matrices is 4096 elements.
Figure 3 Parameter setting when NZ2ND is enabled

Single transfer-in mode

  • ndNum = 2 indicates that the number of source NZ matrices is 2. In the figure, the blue area is NZ matrix 1 and the purple area is NZ matrix 2.
  • nSize = 32 indicates that the source NZ matrix (blue area in the figure) has 32 elements along the N dimension.
  • mSize = 48 indicates that the source NZ matrix has 48 elements along the M dimension.
  • srcStride = 64 indicates the stride between the start addresses of adjacent Z-tiles in the source NZ matrix. The interval between the start addresses of the first and the second blue Z-tiles is 64 × C0_Size.
  • dstStride = 64 indicates that each row in the destination ND matrix contains 64 elements.
  • srcNdStride = 256 indicates that the stride between the start addresses of different NZ matrices is 256, in the unit of C0_Size.
  • dstNdStride = 4096 indicates that the stride between the start addresses of adjacent destination ND matrices is 4096 elements.
Figure 4 Single transfer-in mode settings (NZ2ND enabled)

Dual transfer-in mode

  • ndNum = 2 indicates that the number of source NZ matrices is 2. In the figure, the red area indicates matrix 1, and the blue area indicates matrix 2.
  • nSize = 32 indicates that the source NZ matrix (red or blue area in the figure) has 32 elements along the N dimension.
  • mSize = 48 indicates that the source NZ matrix has 48 elements along the M dimension.
  • srcStride = 64 indicates the stride between the start addresses of adjacent Z-arranged matrices in the source NZ matrix. In the red area of the figure below, the stride between the start address of the light-colored Z-arranged matrix on the left and the start address of the dark-colored Z-arranged matrix on the right is 64 × C0_Size.
  • dstStride = 64 indicates that each row in the destination ND matrix contains 64 elements.
  • ndNum = 2 indicates the number of source NZ matrices.
  • srcNdStride = 240 indicates that the stride between the start addresses of different NZ matrices is 240 × C0_Size.
  • dstNdStride = 4096 indicates that the stride between the start addresses of adjacent destination ND matrices is 4096 elements.
    Figure 5 Dual transfer-in mode settings (NZ2ND enabled)

If NZ2DN is enabled, an example of parameter settings and the description are as follows:

Single transfer mode:

  • dnNum = 2 indicates that the number of source NZ matrices is 2. In the figure below, the blue area is NZ matrix 1, and the purple area is NZ matrix 2.
  • nSize = 32 indicates that the source NZ matrix (blue area in the figure) has 32 elements along the N dimension.
  • mSize = 48 indicates that the source NZ matrix has 48 elements along the M dimension.
  • srcStride = 80 indicates the stride between the start addresses of adjacent Z-arranged matrices in the source NZ matrix. In the figure below, the stride between the start addresses of two adjacent Z-arranged matrices in the blue area is 80 × C0_Size.
  • dstStride = 60 indicates that each row in the destination DN matrix contains 60 elements.
  • ndNum = 2 indicates the number of source NZ matrices.
  • srcNzMatrixStride = 240 indicates the stride between different source NZ matrices. In the figure below, the stride between the start address of the first Z-arranged matrix in the blue area and the start address of the second Z-arranged matrix in the purple area is 240 × C0_Size.
  • srcNzC0Stride = 1 indicates the stride between the addresses of adjacent rows of the NZ fractal in the source matrix.
  • dstDnMatrixStride indicates that the stride between the start addresses of adjacent DN matrices is 48 × 80 = 3840 elements.
Figure 6 Single transfer mode (NZ2DN enabled)

Single transfer mode:

  • dnNum = 2 indicates that the number of source NZ matrices is 2. In the figure below, the blue area indicates NZ matrix 1, and the red area indicates NZ matrix 2.
  • nSize = 24 indicates that the source NZ matrix (blue area in the figure) has 24 elements along the N dimension.
  • mSize = 24 indicates that the source NZ matrix has 24 elements along the M dimension.
  • srcStride = 80 indicates the stride between the start addresses of adjacent Z-arranged matrices in the source NZ matrix. In the figure below, the stride between the start addresses of two adjacent Z-arranged matrices in the blue area is 80 × C0_Size.
  • dstStride = 60 indicates that each row in the destination DN matrix contains 60 elements.
  • ndNum = 2 indicates the number of source NZ matrices.
  • srcNzMatrixStride = 240 indicates the stride between different source NZ matrices. In the figure below, the stride between the start address of the first Z-arranged matrix in the blue area and the start address of the second Z-arranged matrix in the purple area is 240 × C0_Size.
  • srcNzC0Stride = 2 indicates the stride between the addresses of adjacent rows of the NZ fractal in the source matrix.
  • dstDnMatrixStride indicates that the stride between the start addresses of adjacent DN matrices is 48 × 60 = 2880 elements.
Figure 7 Single transfer mode (NZ2DN enabled)

Restrictions

  • ndNum = 0 indicates that this instruction will not be executed and a warning will be reported.
  • If the quantization input is of the float32 data type, the description is as follows:
    • A standard IEEE-754 float32 consists of 1 sign bit, 8 exponent bits, and 23 mantissa bits, while the AI processor supported float32 is composed of 1 sign bit, 8 exponent bits, and 10 mantissa bits.
    • If you use standard IEEE-754 float32 inputs, the API converts the inputs into the float32 format supported by the processor. In this case, if standard IEEE-754 float32 is used during golden data generation, precision mismatch may occur. The lower 13 bits of the 23-bit mantissa of quantization parameters need to be cleared before quantization computation.

Examples

  • Example 1: CO1 -> GM. The input matrices A and B use the data type half, and the output matrix C is also half. By default, Nz2Nd format conversion is enabled, and F322F16 is enabled to cast the mmad computation result from float to half. 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
    		 
    AscendC::GlobalTensor<half> cGM;
AscendC::LocalTensor<half> c1Local = outQueueCO1.DeQue<half>();
uint16_t m = 32, n = 16, k = 32; // m and n indicate the output row and column, respectively.
uint16_t B32_B16_SIZE = 16 * 16;
uint8_t n_block = 16;
AscendC::FixpipeParamsV220 fixpipeParams;
fixpipeParams.nSize = n;
fixpipeParams.mSize = m;
fixpipeParams.srcStride = m;
fixpipeParams.dstStride = n;
fixpipeParams.ndNum = 1;
fixpipeParams.srcNdStride = 2;
fixpipeParams.dstNdStride = m * n;
fixpipeParams.quantPre = QuantMode_t::F322F16; // Enable F322F16 to quantize the result computed by the mmad from float to half.
AscendC::Fixpipe(cGM, c1Local, fixpipeParams);
outQueueCO1.FreeTensor(c1Local);
    Result example: 
    Input matrix A:
 [[6. 3. 9. 4. 5. 3. 9. 7. 3. 6. 2. 7. 3. 8. 8. 1. 8. 8. 5. 6. 6. 8. 2. 2.
  3. 6. 4. 8. 9. 6. 6. 1.]
 [2. 5. 7. 2. 4. 2. 5. 2. 4. 6. 4. 8. 5. 7. 1. 4. 3. 1. 8. 6. 4. 6. 9. 1.
  8. 2. 9. 5. 3. 7. 7. 8.]
 [5. 8. 2. 1. 4. 5. 7. 7. 4. 6. 8. 5. 6. 5. 4. 2. 5. 4. 7. 9. 5. 4. 7. 4.
  2. 2. 1. 7. 8. 4. 6. 6.]
 [8. 2. 4. 7. 6. 9. 7. 7. 4. 5. 6. 7. 6. 6. 5. 3. 7. 6. 7. 4. 5. 4. 1. 9.
  6. 7. 8. 9. 4. 9. 5. 5.]
 [4. 9. 4. 2. 7. 8. 3. 4. 1. 5. 3. 8. 8. 5. 5. 8. 3. 8. 5. 3. 9. 4. 5. 4.
  2. 4. 3. 8. 9. 8. 4. 3.]
 [1. 3. 8. 3. 1. 9. 9. 5. 5. 6. 3. 2. 3. 4. 3. 3. 5. 9. 6. 7. 1. 3. 4. 2.
  8. 5. 9. 1. 9. 5. 8. 9.]
 [3. 3. 1. 3. 5. 2. 7. 8. 8. 9. 6. 9. 3. 6. 5. 5. 2. 3. 2. 3. 5. 1. 6. 1.
  7. 8. 7. 2. 2. 7. 8. 1.]
 [4. 4. 6. 4. 6. 5. 1. 2. 7. 8. 3. 2. 9. 9. 7. 7. 7. 1. 2. 7. 2. 1. 5. 2.
  1. 3. 2. 1. 3. 3. 2. 9.]
 [4. 6. 3. 5. 8. 4. 1. 1. 2. 5. 8. 8. 8. 3. 9. 6. 5. 6. 7. 9. 2. 1. 9. 3.
  2. 5. 4. 1. 7. 5. 3. 9.]
 [7. 2. 3. 4. 9. 5. 6. 3. 4. 5. 4. 7. 4. 1. 9. 4. 2. 1. 7. 4. 9. 2. 4. 5.
  4. 5. 8. 7. 2. 2. 8. 3.]
 [5. 7. 6. 2. 9. 4. 7. 1. 8. 6. 2. 1. 6. 5. 5. 6. 3. 8. 1. 5. 2. 1. 8. 3.
  1. 9. 3. 3. 5. 2. 2. 5.]
 [4. 7. 5. 9. 9. 6. 7. 3. 1. 9. 2. 6. 5. 2. 6. 7. 1. 7. 6. 9. 3. 7. 6. 1.
  3. 9. 2. 4. 1. 9. 4. 8.]
 [2. 4. 3. 1. 1. 2. 2. 7. 2. 3. 7. 9. 8. 8. 3. 4. 1. 2. 9. 2. 9. 4. 4. 8.
  5. 7. 7. 3. 9. 9. 5. 3.]
 [3. 1. 1. 6. 1. 8. 3. 3. 6. 3. 4. 4. 3. 8. 2. 1. 1. 1. 6. 5. 8. 8. 5. 8.
  5. 1. 2. 2. 1. 3. 7. 4.]
 [4. 2. 8. 4. 4. 1. 9. 6. 9. 9. 5. 4. 3. 1. 3. 8. 1. 2. 8. 2. 5. 8. 9. 3.
  2. 5. 9. 7. 7. 4. 2. 1.]
 [2. 6. 7. 1. 3. 9. 9. 9. 6. 4. 5. 8. 1. 3. 7. 3. 8. 7. 3. 4. 8. 6. 9. 6.
  8. 9. 4. 4. 7. 6. 1. 4.]
 [2. 8. 2. 1. 2. 6. 2. 8. 5. 9. 9. 8. 6. 4. 4. 1. 4. 1. 4. 4. 4. 7. 5. 9.
  9. 8. 9. 1. 8. 4. 7. 3.]
 [3. 6. 2. 5. 1. 2. 9. 2. 6. 7. 4. 5. 9. 6. 5. 9. 7. 9. 5. 5. 6. 7. 4. 7.
  7. 6. 3. 6. 5. 2. 8. 3.]
 [1. 7. 3. 2. 4. 8. 1. 7. 3. 4. 1. 6. 1. 4. 4. 1. 6. 7. 9. 3. 9. 2. 2. 2.
  2. 8. 1. 1. 6. 3. 6. 1.]
 [4. 3. 9. 5. 2. 2. 1. 8. 5. 8. 9. 2. 4. 3. 2. 1. 8. 6. 6. 2. 9. 2. 9. 3.
  9. 5. 3. 7. 9. 7. 6. 2.]
 [9. 4. 8. 1. 3. 7. 9. 5. 2. 4. 9. 9. 6. 9. 6. 4. 6. 3. 3. 9. 6. 8. 1. 5.
  5. 1. 6. 5. 1. 9. 3. 9.]
 [2. 5. 2. 1. 8. 9. 9. 8. 1. 6. 1. 1. 9. 8. 3. 5. 6. 4. 2. 1. 3. 7. 8. 9.
  6. 6. 1. 9. 1. 7. 6. 8.]
 [4. 7. 6. 6. 2. 2. 1. 8. 7. 1. 1. 2. 1. 1. 9. 8. 9. 4. 9. 5. 7. 8. 9. 9.
  5. 1. 6. 8. 9. 6. 7. 5.]
 [1. 1. 6. 9. 9. 3. 7. 6. 5. 6. 5. 1. 5. 5. 3. 7. 6. 7. 4. 8. 8. 2. 2. 5.
  7. 8. 8. 2. 9. 1. 5. 1.]
 [5. 4. 6. 8. 8. 3. 7. 7. 5. 7. 8. 7. 4. 8. 2. 9. 4. 8. 1. 3. 8. 5. 3. 7.
  3. 7. 1. 9. 1. 5. 4. 7.]
 [6. 3. 1. 2. 8. 3. 2. 6. 8. 2. 8. 4. 1. 9. 4. 7. 5. 1. 7. 5. 5. 1. 1. 1.
  2. 8. 1. 7. 9. 8. 5. 4.]
 [2. 8. 5. 1. 3. 4. 9. 8. 6. 9. 6. 2. 4. 2. 2. 7. 8. 2. 1. 3. 7. 1. 4. 6.
  4. 6. 3. 3. 1. 6. 8. 3.]
 [5. 1. 5. 5. 9. 7. 9. 2. 1. 4. 7. 8. 1. 9. 8. 1. 2. 4. 3. 9. 9. 6. 7. 9.
  1. 5. 1. 9. 2. 5. 6. 9.]
 [1. 9. 9. 6. 5. 7. 9. 5. 4. 1. 2. 8. 3. 8. 1. 9. 6. 1. 7. 9. 3. 2. 2. 4.
  7. 9. 9. 4. 7. 1. 5. 8.]
 [3. 2. 2. 5. 9. 3. 6. 9. 2. 4. 4. 8. 4. 2. 6. 1. 2. 8. 8. 8. 9. 7. 7. 1.
  9. 6. 5. 8. 3. 3. 3. 4.]
 [9. 1. 6. 1. 3. 7. 8. 1. 2. 6. 5. 9. 4. 4. 7. 2. 3. 9. 8. 7. 8. 2. 6. 4.
  5. 6. 5. 4. 9. 6. 1. 9.]
 [4. 3. 2. 7. 8. 1. 7. 2. 9. 7. 7. 4. 2. 8. 2. 5. 6. 9. 5. 1. 3. 9. 8. 2.
  4. 8. 4. 7. 4. 1. 3. 7.]]
Input matrix B:
[[3. 5. 9. 6. 2. 9. 3. 6. 5. 9. 5. 5. 3. 8. 5. 2.]
 [5. 1. 5. 7. 5. 4. 2. 2. 4. 8. 1. 1. 3. 3. 7. 2.]
 [6. 7. 4. 6. 1. 4. 8. 3. 9. 2. 2. 3. 4. 6. 5. 3.]
 [4. 8. 2. 6. 4. 8. 6. 7. 3. 8. 6. 7. 3. 8. 1. 1.]
 [6. 7. 8. 6. 1. 9. 9. 3. 9. 9. 2. 1. 3. 3. 3. 3.]
 [7. 2. 4. 7. 5. 8. 9. 2. 1. 7. 9. 6. 8. 7. 1. 3.]
 [3. 3. 9. 2. 3. 9. 4. 1. 8. 2. 5. 1. 2. 6. 5. 5.]
 [6. 4. 8. 8. 7. 5. 9. 6. 7. 6. 8. 8. 2. 6. 1. 2.]
 [4. 2. 3. 8. 6. 1. 1. 1. 7. 9. 5. 2. 2. 5. 7. 6.]
 [4. 5. 9. 5. 6. 8. 1. 2. 1. 9. 2. 7. 8. 6. 6. 1.]
 [4. 8. 6. 6. 3. 1. 7. 8. 7. 3. 2. 9. 8. 6. 9. 8.]
 [3. 2. 5. 5. 7. 9. 7. 7. 4. 8. 3. 5. 2. 7. 1. 2.]
 [3. 8. 2. 8. 9. 5. 1. 5. 7. 4. 1. 3. 4. 1. 4. 6.]
 [9. 5. 2. 2. 4. 6. 3. 3. 7. 1. 9. 6. 8. 6. 4. 7.]
 [2. 3. 8. 1. 5. 9. 8. 4. 5. 4. 6. 5. 4. 5. 3. 2.]
 [3. 5. 4. 2. 1. 2. 9. 2. 3. 8. 9. 8. 8. 1. 2. 7.]
 [1. 4. 5. 1. 3. 8. 2. 5. 9. 9. 5. 5. 5. 6. 4. 2.]
 [7. 6. 7. 7. 6. 9. 1. 3. 8. 1. 9. 8. 8. 5. 1. 6.]
 [5. 3. 8. 9. 8. 2. 6. 6. 1. 3. 2. 1. 2. 9. 3. 9.]
 [1. 1. 4. 9. 8. 6. 6. 5. 6. 8. 4. 2. 2. 7. 2. 1.]
 [8. 1. 3. 5. 8. 7. 5. 7. 4. 6. 7. 4. 8. 2. 2. 3.]
 [5. 8. 6. 8. 1. 8. 6. 8. 3. 9. 1. 1. 3. 8. 3. 2.]
 [7. 7. 5. 1. 5. 4. 6. 1. 1. 6. 8. 8. 1. 7. 7. 2.]
 [1. 7. 7. 7. 7. 6. 1. 7. 3. 3. 8. 9. 3. 8. 9. 8.]
 [4. 9. 5. 6. 9. 6. 8. 9. 1. 1. 6. 5. 1. 4. 3. 5.]
 [4. 1. 8. 9. 6. 5. 5. 7. 8. 9. 8. 2. 7. 5. 5. 3.]
 [9. 8. 4. 9. 5. 4. 7. 5. 7. 6. 9. 8. 5. 7. 2. 9.]
 [6. 6. 5. 1. 4. 5. 9. 6. 7. 5. 5. 2. 3. 7. 6. 5.]
 [5. 2. 5. 7. 9. 2. 2. 3. 2. 3. 1. 4. 6. 5. 3. 1.]
 [5. 1. 9. 3. 2. 4. 1. 6. 7. 7. 4. 9. 8. 8. 6. 1.]
 [3. 7. 5. 6. 7. 8. 2. 2. 8. 7. 6. 1. 3. 5. 3. 2.]
 [7. 6. 7. 8. 6. 5. 2. 2. 8. 2. 2. 6. 6. 4. 9. 6.]]
Output matrix C:
[[ 807.  767. 1007.  925.  853. 1079.  837.  782.  977.  960.  838.  746.
   767. 1013.  642.  594.]
 [ 778.  775.  850.  874.  801.  853.  767.  682.  808.  852.  719.  709.
   651.  891.  663.  635.]
 [ 734.  705.  927.  901.  865.  906.  742.  687.  840.  892.  725.  718.
   692.  911.  702.  601.]
 [ 877.  895. 1099. 1070.  954. 1136.  926.  912. 1028. 1057.  983.  930.
   859. 1119.  760.  768.]
 [ 818.  722.  931.  904.  857.  969.  809.  724.  846.  948.  812.  786.
   811.  885.  644.  619.]
 [ 780.  750.  907.  964.  865.  905.  738.  638.  861.  808.  816.  759.
   735.  913.  627.  640.]
 [ 697.  671.  865.  810.  780.  863.  729.  656.  803.  892.  798.  734.
   664.  819.  593.  561.]
 [ 619.  633.  716.  734.  667.  767.  612.  515.  749.  794.  641.  652.
   650.  705.  596.  518.]
 [ 716.  738.  908.  907.  838.  902.  767.  684.  829.  907.  726.  787.
   728.  872.  671.  609.]
 [ 692.  710.  876.  838.  779.  926.  812.  692.  791.  894.  767.  660.
   629.  844.  588.  597.]
 [ 671.  639.  812.  787.  684.  815.  637.  511.  806.  819.  714.  627.
   652.  734.  628.  546.]
 [ 779.  764. 1011.  962.  806. 1042.  845.  728.  883. 1027.  794.  762.
   764.  949.  667.  576.]
 [ 750.  690.  856.  907.  875.  801.  716.  772.  771.  803.  760.  772.
   724.  865.  633.  656.]
 [ 598.  605.  649.  731.  678.  741.  591.  593.  577.  694.  662.  591.
   536.  750.  508.  508.]
 [ 754.  750.  902.  869.  746.  815.  807.  669.  780.  912.  750.  719.
   658.  905.  658.  633.]
 [ 844.  758. 1037.  971.  920. 1038.  903.  800.  920.  983.  937.  863.
   791. 1011.  726.  648.]
 [ 754.  782.  935. 1018.  936.  909.  770.  795.  799.  947.  796.  811.
   726.  937.  708.  644.]
 [ 744.  828.  940.  936.  914. 1014.  753.  760.  893.  946.  874.  777.
   768.  920.  699.  706.]
 [ 615.  467.  719.  754.  714.  750.  601.  560.  637.  739.  650.  544.
   598.  699.  434.  437.]
 [ 785.  791.  906.  889.  868.  866.  766.  768.  836.  871.  787.  814.
   738.  920.  693.  592.]
 [ 814.  822. 1006.  963.  831. 1062.  868.  826.  991.  950.  834.  853.
   809. 1021.  745.  700.]
 [ 782.  812.  957.  847.  800.  998.  773.  688.  882.  890.  854.  770.
   730.  889.  721.  642.]
 [ 792.  815.  966.  947.  895.  942.  858.  786.  859.  995.  884.  827.
   701. 1006.  711.  657.]
 [ 758.  791.  878.  960.  861.  938.  818.  735.  889.  906.  861.  763.
   751.  869.  588.  649.]
 [ 830.  853.  990.  936.  817. 1044.  862.  796.  990.  994.  902.  865.
   834.  953.  744.  698.]
 [ 679.  586.  833.  792.  716.  754.  713.  653.  816.  856.  708.  654.
   698.  802.  608.  566.]
 [ 636.  642.  844.  775.  723.  821.  652.  600.  809.  864.  743.  693.
   671.  763.  652.  546.]
 [ 804.  789.  987.  887.  824. 1084.  868.  766.  933.  924.  859.  786.
   762. 1002.  735.  639.]
 [ 813.  765.  906. 1016.  889.  947.  902.  735.  933.  949.  870.  738.
   737.  943.  664.  708.]
 [ 790.  769.  946.  935.  877.  996.  899.  798.  840.  903.  807.  718.
   651.  919.  579.  605.]
 [ 803.  725. 1003.  949.  900. 1002.  792.  749.  860.  863.  818.  812.
   790.  972.  686.  657.]
 [ 787.  813.  910.  873.  751.  927.  751.  688.  874.  914.  795.  733.
   721.  903.  697.  664.]]
  • Example 2: path CO1->GM, tensor quantization enabled. The data type of matrix A and matrix B is int8, and the data type of matrix C is half. By default, NZ2ND format conversion is enabled, and tensor quantization (VDEQF16) is enabled to quantize the mmad computation result from int32 to half. 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
    		 
    AscendC::GlobalTensor<half> cGM;
AscendC::LocalTensor<dstCO1_T> c1Local = outQueueCO1.DeQue<dstCO1_T>();
AscendC::LocalTensor<uint64_t> deqTensorLocal = deqQueue.DeQue<uint64_t>();
uint16_t m = 32, n = 16, k = 32;
uint16_t B32_B16_SIZE = 16 * 16;
uint8_t n_block = 16;
AscendC::FixpipeParamsV220 fixpipeParams;
fixpipeParams.nSize = n;
fixpipeParams.mSize = m;
fixpipeParams.srcStride = m;
fixpipeParams.dstStride = n;
fixpipeParams.ndNum = 1;
fixpipeParams.srcNdStride = 4;
fixpipeParams.dstNdStride = m * n;
fixpipeParams.quantPre = QuantMode_t::VDEQF16;
        AscendC::Fixpipe(cGM, c1Local, deqTensorLocal, fixpipeParams); // NZ2ND conversion can be performed from CO1 to GM.
outQueueCO1.FreeTensor(c1Local);
deqQueue.FreeTensor(deqTensorLocal);
    Result example: 
    Input matrix A:
[[6 3 9 4 5 3 9 7 3 6 2 7 3 8 8 1 8 8 5 6 6 8 2 2 3 6 4 8 9 6 6 1]
 [2 5 7 2 4 2 5 2 4 6 4 8 5 7 1 4 3 1 8 6 4 6 9 1 8 2 9 5 3 7 7 8]
 [5 8 2 1 4 5 7 7 4 6 8 5 6 5 4 2 5 4 7 9 5 4 7 4 2 2 1 7 8 4 6 6]
 [8 2 4 7 6 9 7 7 4 5 6 7 6 6 5 3 7 6 7 4 5 4 1 9 6 7 8 9 4 9 5 5]
 [4 9 4 2 7 8 3 4 1 5 3 8 8 5 5 8 3 8 5 3 9 4 5 4 2 4 3 8 9 8 4 3]
 [1 3 8 3 1 9 9 5 5 6 3 2 3 4 3 3 5 9 6 7 1 3 4 2 8 5 9 1 9 5 8 9]
 [3 3 1 3 5 2 7 8 8 9 6 9 3 6 5 5 2 3 2 3 5 1 6 1 7 8 7 2 2 7 8 1]
 [4 4 6 4 6 5 1 2 7 8 3 2 9 9 7 7 7 1 2 7 2 1 5 2 1 3 2 1 3 3 2 9]
 [4 6 3 5 8 4 1 1 2 5 8 8 8 3 9 6 5 6 7 9 2 1 9 3 2 5 4 1 7 5 3 9]
 [7 2 3 4 9 5 6 3 4 5 4 7 4 1 9 4 2 1 7 4 9 2 4 5 4 5 8 7 2 2 8 3]
 [5 7 6 2 9 4 7 1 8 6 2 1 6 5 5 6 3 8 1 5 2 1 8 3 1 9 3 3 5 2 2 5]
 [4 7 5 9 9 6 7 3 1 9 2 6 5 2 6 7 1 7 6 9 3 7 6 1 3 9 2 4 1 9 4 8]
 [2 4 3 1 1 2 2 7 2 3 7 9 8 8 3 4 1 2 9 2 9 4 4 8 5 7 7 3 9 9 5 3]
 [3 1 1 6 1 8 3 3 6 3 4 4 3 8 2 1 1 1 6 5 8 8 5 8 5 1 2 2 1 3 7 4]
 [4 2 8 4 4 1 9 6 9 9 5 4 3 1 3 8 1 2 8 2 5 8 9 3 2 5 9 7 7 4 2 1]
 [2 6 7 1 3 9 9 9 6 4 5 8 1 3 7 3 8 7 3 4 8 6 9 6 8 9 4 4 7 6 1 4]
 [2 8 2 1 2 6 2 8 5 9 9 8 6 4 4 1 4 1 4 4 4 7 5 9 9 8 9 1 8 4 7 3]
 [3 6 2 5 1 2 9 2 6 7 4 5 9 6 5 9 7 9 5 5 6 7 4 7 7 6 3 6 5 2 8 3]
 [1 7 3 2 4 8 1 7 3 4 1 6 1 4 4 1 6 7 9 3 9 2 2 2 2 8 1 1 6 3 6 1]
 [4 3 9 5 2 2 1 8 5 8 9 2 4 3 2 1 8 6 6 2 9 2 9 3 9 5 3 7 9 7 6 2]
 [9 4 8 1 3 7 9 5 2 4 9 9 6 9 6 4 6 3 3 9 6 8 1 5 5 1 6 5 1 9 3 9]
 [2 5 2 1 8 9 9 8 1 6 1 1 9 8 3 5 6 4 2 1 3 7 8 9 6 6 1 9 1 7 6 8]
 [4 7 6 6 2 2 1 8 7 1 1 2 1 1 9 8 9 4 9 5 7 8 9 9 5 1 6 8 9 6 7 5]
 [1 1 6 9 9 3 7 6 5 6 5 1 5 5 3 7 6 7 4 8 8 2 2 5 7 8 8 2 9 1 5 1]
 [5 4 6 8 8 3 7 7 5 7 8 7 4 8 2 9 4 8 1 3 8 5 3 7 3 7 1 9 1 5 4 7]
 [6 3 1 2 8 3 2 6 8 2 8 4 1 9 4 7 5 1 7 5 5 1 1 1 2 8 1 7 9 8 5 4]
 [2 8 5 1 3 4 9 8 6 9 6 2 4 2 2 7 8 2 1 3 7 1 4 6 4 6 3 3 1 6 8 3]
 [5 1 5 5 9 7 9 2 1 4 7 8 1 9 8 1 2 4 3 9 9 6 7 9 1 5 1 9 2 5 6 9]
 [1 9 9 6 5 7 9 5 4 1 2 8 3 8 1 9 6 1 7 9 3 2 2 4 7 9 9 4 7 1 5 8]
 [3 2 2 5 9 3 6 9 2 4 4 8 4 2 6 1 2 8 8 8 9 7 7 1 9 6 5 8 3 3 3 4]
 [9 1 6 1 3 7 8 1 2 6 5 9 4 4 7 2 3 9 8 7 8 2 6 4 5 6 5 4 9 6 1 9]
 [4 3 2 7 8 1 7 2 9 7 7 4 2 8 2 5 6 9 5 1 3 9 8 2 4 8 4 7 4 1 3 7]]
Input matrix B:
[[3 5 9 6 2 9 3 6 5 9 5 5 3 8 5 2 5 1 5 7 5 4 2 2 4 8 1 1 3 3 7 2]
 [6 7 4 6 1 4 8 3 9 2 2 3 4 6 5 3 4 8 2 6 4 8 6 7 3 8 6 7 3 8 1 1]
 [6 7 8 6 1 9 9 3 9 9 2 1 3 3 3 3 7 2 4 7 5 8 9 2 1 7 9 6 8 7 1 3]
 [3 3 9 2 3 9 4 1 8 2 5 1 2 6 5 5 6 4 8 8 7 5 9 6 7 6 8 8 2 6 1 2]
 [4 2 3 8 6 1 1 1 7 9 5 2 2 5 7 6 4 5 9 5 6 8 1 2 1 9 2 7 8 6 6 1]
 [4 8 6 6 3 1 7 8 7 3 2 9 8 6 9 8 3 2 5 5 7 9 7 7 4 8 3 5 2 7 1 2]
 [3 8 2 8 9 5 1 5 7 4 1 3 4 1 4 6 9 5 2 2 4 6 3 3 7 1 9 6 8 6 4 7]
 [2 3 8 1 5 9 8 4 5 4 6 5 4 5 3 2 3 5 4 2 1 2 9 2 3 8 9 8 8 1 2 7]
 [1 4 5 1 3 8 2 5 9 9 5 5 5 6 4 2 7 6 7 7 6 9 1 3 8 1 9 8 8 5 1 6]
 [5 3 8 9 8 2 6 6 1 3 2 1 2 9 3 9 1 1 4 9 8 6 6 5 6 8 4 2 2 7 2 1]
 [8 1 3 5 8 7 5 7 4 6 7 4 8 2 2 3 5 8 6 8 1 8 6 8 3 9 1 1 3 8 3 2]
 [7 7 5 1 5 4 6 1 1 6 8 8 1 7 7 2 1 7 7 7 7 6 1 7 3 3 8 9 3 8 9 8]
 [4 9 5 6 9 6 8 9 1 1 6 5 1 4 3 5 4 1 8 9 6 5 5 7 8 9 8 2 7 5 5 3]
 [9 8 4 9 5 4 7 5 7 6 9 8 5 7 2 9 6 6 5 1 4 5 9 6 7 5 5 2 3 7 6 5]
 [5 2 5 7 9 2 2 3 2 3 1 4 6 5 3 1 5 1 9 3 2 4 1 6 7 7 4 9 8 8 6 1]
 [3 7 5 6 7 8 2 2 8 7 6 1 3 5 3 2 7 6 7 8 6 5 2 2 8 2 2 6 6 4 9 6]
 [4 8 4 7 6 4 1 5 1 7 2 4 1 1 5 5 3 5 2 2 7 5 4 7 5 8 2 4 6 2 8 9]
 [9 2 7 4 1 7 4 4 7 1 9 7 4 5 3 8 7 8 8 4 1 9 9 8 4 9 3 1 1 8 6 3]
 [4 9 2 7 3 9 5 2 6 8 8 7 1 5 6 1 9 4 1 6 1 6 2 1 3 5 2 6 6 8 1 9]
 [8 3 9 4 9 7 7 4 2 8 4 1 7 9 3 9 1 3 8 7 6 1 4 9 1 6 8 7 6 3 2 2]
 [2 3 4 5 4 9 9 3 4 4 7 3 8 7 9 7 7 5 8 5 8 4 1 8 1 9 5 8 8 3 9 5]
 [7 7 5 6 6 1 4 7 9 7 6 2 3 5 7 1 3 5 9 2 2 4 6 9 4 5 9 7 2 3 8 3]
 [2 9 2 4 1 4 7 2 5 4 8 8 2 3 3 3 1 3 5 9 5 8 3 8 6 8 4 1 1 6 1 7]
 [7 1 8 5 2 6 6 6 7 1 7 4 2 1 5 9 6 4 2 8 4 3 2 5 9 1 3 9 1 9 3 9]
 [9 4 4 9 4 9 4 5 4 1 3 2 6 5 6 1 8 2 4 1 7 5 9 3 5 7 9 3 9 4 1 4]
 [1 6 2 1 7 1 5 2 8 8 6 4 4 2 5 2 5 8 1 2 9 3 1 1 8 6 9 4 2 2 1 8]
 [9 1 8 3 8 7 1 6 2 3 8 1 4 8 6 7 4 8 5 9 3 7 4 1 3 8 4 3 3 3 2 4]
 [9 4 5 6 2 2 3 7 2 2 3 3 2 8 5 4 5 5 5 5 1 5 8 4 4 1 1 3 8 5 3 8]
 [6 3 6 7 9 9 4 5 9 2 6 6 4 9 9 2 8 9 4 7 4 7 4 4 6 8 9 6 2 7 3 6]
 [9 1 5 8 8 8 5 9 6 8 4 9 4 2 3 6 2 2 4 8 2 6 6 4 6 7 6 9 5 8 5 9]
 [5 5 5 9 2 4 6 3 1 5 2 2 8 6 3 2 6 2 7 8 7 9 6 2 6 6 1 5 1 3 4 7]
 [6 6 9 1 2 3 4 1 1 5 3 2 3 4 5 5 3 8 6 6 9 1 5 9 2 2 9 4 4 6 2 2]]
Input quantization tensor:
[1065353216 1073741824 1065353216 1073741824 1065353216 1065353216
 1065353216 1073741824 1073741824 1073741824 1065353216 1065353216
 1065353216 1065353216 1065353216 1073741824 1073741824 1065353216
 1073741824 1065353216 1073741824 1073741824 1065353216 1065353216
 1073741824 1065353216 1073741824 1073741824 1065353216 1073741824
 1065353216 1073741824]
Output matrix C:
[[ 943. 1676.  932. 1962.  893.  941.  817. 1528. 1778. 1740.  823.  715.
   659.  915.  818. 1500. 1710.  794. 1824.  890. 1558. 1938.  846.  827.
  1596. 1066. 1916. 1842.  822. 1860.  724. 1702.]
 [ 889. 1638.  814. 1730.  757.  863.  772. 1326. 1454. 1592.  780.  620.
   582.  821.  720. 1326. 1430.  715. 1632.  930. 1534. 1790.  751.  762.
  1380.  921. 1736. 1546.  721. 1712.  564. 1524.]
 [ 855. 1614.  847. 1774.  805.  873.  817. 1442. 1548. 1544.  776.  690.
   638.  849.  744. 1416. 1486.  755. 1668.  927. 1472. 1798.  750.  853.
  1456.  984. 1682. 1630.  731. 1800.  596. 1530.]
 [1033. 1746. 1044. 2034.  940. 1044.  873. 1764. 1860. 1816.  931.  802.
   717.  951.  910. 1742. 1832.  857. 1934. 1053. 1770. 2082.  904.  883.
  1818. 1126. 1934. 1972.  867. 2074.  729. 1890.]
 [ 902. 1650.  872. 1874.  821.  897.  850. 1482. 1736. 1530.  846.  746.
   632.  897.  830. 1496. 1582.  793. 1814.  976. 1564. 1954.  770.  851.
  1546. 1058. 1686. 1766.  749. 1930.  715. 1588.]
 [ 886. 1578.  900. 1740.  799.  913.  756. 1410. 1630. 1492.  737.  643.
   666.  819.  749. 1458. 1612.  762. 1596.  893. 1574. 1878.  832.  759.
  1494.  979. 1866. 1572.  703. 1750.  503. 1498.]
 [ 753. 1364.  754. 1576.  802.  818.  702. 1262. 1416. 1494.  746.  617.
   612.  775.  655. 1254. 1380.  690. 1578.  845. 1496. 1734.  663.  659.
  1500.  908. 1638. 1544.  693. 1566.  569. 1492.]
 [ 677. 1428.  767. 1478.  708.  704.  662. 1154. 1298. 1428.  627.  533.
   502.  709.  580. 1288. 1192.  585. 1526.  810. 1478. 1478.  617.  716.
  1342.  833. 1472. 1348.  647. 1508.  521. 1106.]
 [ 851. 1560.  858. 1662.  837.  854.  766. 1264. 1496. 1588.  813.  677.
   589.  821.  730. 1388. 1402.  758. 1792.  994. 1588. 1796.  673.  863.
  1472. 1029. 1650. 1616.  687. 1884.  613. 1378.]
 [ 751. 1388.  793. 1644.  755.  802.  683. 1236. 1374. 1494.  723.  569.
   600.  811.  750. 1276. 1482.  652. 1674.  888. 1500. 1702.  591.  673.
  1378.  906. 1442. 1632.  739. 1614.  605. 1420.]
 [ 683. 1436.  740. 1504.  696.  720.  652. 1160. 1588. 1438.  681.  568.
   526.  711.  630. 1306. 1376.  683. 1508.  816. 1456. 1684.  607.  682.
  1422.  866. 1542. 1366.  643. 1590.  511. 1224.]
 [ 873. 1678.  919. 1798.  854.  850.  814. 1350. 1750. 1726.  784.  651.
   619.  864.  775. 1522. 1492.  748. 1870.  977. 1714. 1850.  789.  857.
  1558. 1029. 1886. 1812.  750. 1896.  632. 1446.]
 [ 854. 1464.  787. 1644.  810.  922.  822. 1400. 1542. 1450.  872.  707.
   599.  785.  745. 1294. 1520.  757. 1536.  902. 1398. 1682.  690.  730.
  1500.  946. 1704. 1658.  676. 1736.  611. 1680.]
 [ 657. 1252.  676. 1350.  557.  690.  661. 1132. 1282. 1196.  651.  539.
   538.  654.  614. 1168. 1210.  530. 1388.  705. 1246. 1370.  597.  674.
  1216.  711. 1338. 1362.  524. 1372.  470. 1212.]
 [ 761. 1524.  814. 1636.  805.  906.  706. 1358. 1718. 1606.  797.  590.
   549.  813.  730. 1230. 1568.  737. 1604.  945. 1396. 1830.  676.  670.
  1516.  895. 1726. 1626.  744. 1676.  560. 1574.]
 [ 912. 1756.  910. 1832.  874.  961.  873. 1544. 1906. 1696.  859.  785.
   715.  847.  875. 1508. 1694.  861. 1762.  916. 1704. 2014.  818.  901.
  1670. 1089. 2064. 1926.  836. 1946.  666. 1806.]
 [ 903. 1526.  879. 1748.  865.  887.  848. 1536. 1604. 1480.  834.  677.
   672.  853.  800. 1386. 1490.  792. 1634.  954. 1610. 1864.  768.  811.
  1610. 1047. 1858. 1710.  677. 1794.  566. 1592.]
 [ 908. 1756.  893. 1928.  866.  944.  805. 1522. 1728. 1538.  847.  664.
   653.  868.  779. 1504. 1772.  805. 1832.  954. 1686. 1930.  801.  870.
  1814.  986. 1836. 1724.  773. 1860.  711. 1700.]
 [ 610. 1272.  634. 1334.  578.  681.  674.  988. 1342. 1236.  636.  585.
   520.  666.  652. 1082. 1238.  615. 1248.  652. 1246. 1472.  570.  612.
  1110.  836. 1324. 1412.  551. 1374.  483. 1278.]
 [ 853. 1486.  856. 1790.  754.  997.  838. 1456. 1616. 1528.  807.  674.
   638.  819.  749. 1328. 1606.  731. 1614.  937. 1520. 1904.  841.  777.
  1492. 1082. 1710. 1552.  756. 1740.  560. 1640.]
 [1024. 1736.  989. 1946.  916.  966.  862. 1676. 1646. 1832.  833.  722.
   712.  886.  804. 1638. 1594.  783. 1904.  970. 1644. 1860.  852.  933.
  1534. 1041. 1912. 1826.  846. 1946.  753. 1588.]
 [ 853. 1726.  833. 1888.  777.  757.  798. 1534. 1634. 1460.  752.  692.
   594.  749.  748. 1548. 1490.  705. 1644.  850. 1588. 1772.  818.  816.
  1664.  945. 1706. 1618.  753. 1764.  625. 1636.]
 [ 903. 1646.  959. 1848.  781. 1035.  813. 1446. 1828. 1662.  849.  684.
   647.  892.  839. 1332. 1736.  803. 1822. 1004. 1540. 1914.  792.  840.
  1662. 1018. 1802. 1992.  818. 1854.  663. 1820.]
 [ 827. 1442.  887. 1760.  882.  972.  749. 1342. 1744. 1552.  826.  570.
   655.  850.  779. 1530. 1724.  791. 1758.  908. 1654. 1836.  766.  737.
  1568. 1034. 1812. 1700.  781. 1676.  603. 1512.]
 [ 915. 1642.  953. 1814.  825.  944.  842. 1466. 1836. 1736.  883.  674.
   656.  868.  787. 1622. 1698.  852. 1922.  973. 1722. 1918.  853.  875.
  1672.  999. 1836. 1810.  809. 1922.  733. 1656.]
 [ 742. 1342.  725. 1580.  765.  819.  656. 1236. 1544. 1652.  739.  639.
   592.  770.  681. 1164. 1454.  732. 1506.  794. 1358. 1612.  621.  641.
  1382.  857. 1456. 1548.  704. 1552.  585. 1500.]
 [ 699. 1408.  751. 1612.  729.  795.  720. 1298. 1438. 1414.  632.  540.
   590.  674.  633. 1310. 1380.  656. 1392.  826. 1484. 1658.  670.  675.
  1440.  871. 1522. 1530.  697. 1508.  541. 1466.]
 [ 932. 1604.  911. 1844.  817.  824.  835. 1416. 1644. 1710.  826.  701.
   693.  857.  806. 1668. 1560.  768. 1910.  937. 1660. 1810.  759.  924.
  1522.  963. 1734. 1828.  760. 1958.  697. 1582.]
 [ 909. 1844.  923. 1772.  851.  962.  825. 1330. 1844. 1736.  823.  639.
   662.  889.  841. 1492. 1742.  884. 1674.  940. 1800. 1892.  809.  782.
  1574.  966. 2034. 1866.  814. 1826.  592. 1686.]
 [ 861. 1508.  839. 1670.  806.  884.  777. 1308. 1542. 1538.  838.  650.
   627.  865.  799. 1362. 1530.  753. 1824.  848. 1496. 1744.  755.  811.
  1362. 1018. 1798. 1700.  809. 1690.  628. 1524.]
 [ 916. 1632.  918. 1792.  847.  948.  807. 1450. 1622. 1644.  848.  752.
   655.  883.  830. 1530. 1636.  784. 1750.  959. 1636. 1852.  725.  860.
  1498. 1032. 1818. 1660.  752. 1950.  662. 1574.]
 [ 822. 1602.  807. 1662.  757.  812.  678. 1306. 1734. 1624.  840.  633.
   568.  804.  737. 1366. 1586.  830. 1734.  860. 1544. 1862.  747.  801.
  1578.  921. 1696. 1490.  689. 1740.  622. 1506.]]
  • Example 3: path CO1 -> C1, with tensor quantization enabled. The input matrices A and B use the data type half, and the output matrix C is float. ND2DN is enabled, and quantization is disabled to quantize the mmad computation result from float to float. 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
    		 
    AscendC::LocalTensor<l1out_T> dst_l0c = outQueueCO1.DeQue<l1out_T>();
AscendC::LocalTensor<uint64_t> cbufWorkspace = deqQueue.DeQue<uint64_t>();
uint16_t deqDataSize = AscendC::DivCeil(deq_size * sizeof(uint64_t), 128) * 128;
float tmp = 0.5;
uint64_t val = static_cast<uint64_t>(*reinterpret_cast<int32_t*>(&tmp));
AscendC::FixpipeParamsArch3510<AscendC::CO2Layout::COLUMN_MAJOR> fixpipeParams = {n, m, static_cast<uint16_t>(AscendC::AlignUp(m, AscendC::BLOCK_CUBE)), m};
fixpipeParams.params = {1, 0, 0, 1};
fixpipeParams.reluEn = 1;
AscendC::Fixpipe<dst_T, l1out_T, AscendC::CFG_COLUMN_MAJOR>(output_gm, dst_l0c, fixpipeParams);
outQueueCO1.FreeTensor(dst_l0c);
deqQueue.FreeTensor(cbufWorkspace);
  • Example 4: path CO1 -> C1, with tensor quantization enabled. The input matrices A and B use the data type half, and the output matrix C is float. Both ND2DN and F322F16 are enabled to quantize the mmad computation result from float to half. 
     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
    		 
    AscendC::LocalTensor<l1out_T> dst_l0c = outQueueCO1.DeQue<l1out_T>();
AscendC::LocalTensor<uint64_t> cbufWorkspace = deqQueue.DeQue<uint64_t>();
uint16_t deqDataSize = AscendC::DivCeil(deq_size * sizeof(uint64_t), 128) * 128;
float tmp = 0.5;
uint64_t val = static_cast<uint64_t>(*reinterpret_cast<int32_t*>(&tmp));
AscendC::FixpipeParamsArch3510<AscendC::CO2Layout::COLUMN_MAJOR> fixpipeParams = {n, m, static_cast<uint16_t>(AscendC::AlignUp(m, AscendC::BLOCK_CUBE)), m};
fixpipeParams.params = {1, 0, 0, 1};
fixpipeParams.reluEn = 1;
fixpipeParams.quantPre = F322F16;
AscendC::Fixpipe<dst_T, l1out_T, AscendC::CFG_COLUMN_MAJOR>(output_gm, dst_l0c, fixpipeParams);
outQueueCO1.FreeTensor(dst_l0c);
deqQueue.FreeTensor(cbufWorkspace);
Parent topic: Data Movement
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details