TransDataTo5HD

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product / Atlas A3 inference product	√
Atlas A2 training product / Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product AI Core	√
Atlas inference product Vector Core	x
Atlas training product	√

Function Usage

Converts the NCHW format to the NC1HWC0 format. It can also be used for transposing a two-dimensional matrix data block. For matrix transpose operations, the Transpose API only supports 16 × 16 matrices. This API can process 512 bytes (16 data blocks) per repeat iteration. It supports matrix transposition for various shapes depending on the data type. For example, a 16 × 16 matrix can be transposed within one repeat when the data type is half. Multiple repeat iterations are also supported.

The conversion rules in a single repeat are as follows:

When the input data type is 16-bit wide, each data block holds 16 elements. The instruction runs 16 internal loops. In each iteration, it fetches values from the corresponding positions of the 16 specified data blocks, assembles a new data block, and writes it to the destination address. As shown in the following figure, srcList[0] to srcList[15] indicate 16 data blocks of the source operand.
Figure 1 Conversion rules when the input data type is 16-bit wide
When the input data type is 32-bit wide, each data block holds 8 elements. The instruction runs 8 internal loops. In each iteration, it fetches values from the corresponding positions of the 16 specified data blocks, assembles two new data blocks, and writes them to the destination address. See the following figure.
Figure 2 Conversion rules when the input data type is 32-bit wide
When the input data type is 8-bit wide, each data block holds 32 elements. The instruction runs 16 internal loops. In each iteration, it fetches values from the corresponding positions of the 16 specified data blocks, assembles half a data block, and writes it to the destination address. The srcHighHalf and dstHighHalf parameters determine whether reading and writing operate on the upper half or lower half of the data block. See the following figure.
Figure 3 Conversion rules when the input data type is 8-bit wide

Based on the preceding conversion rules, this API is used to convert the NC1HWC0 format or transpose a matrix. The NC1HWC0 format conversion is complex. The following describes how to convert the NC1HWC0 format:

When the NCHW format is converted to the NC1HWC0 format, if the bit width of the data type is 32 or 16 bits, C0 is 16; if the bit width of the data type is 8 bits, C0 is 32. The following figure uses C0 = 16 as an example.

Prototype

dstList and srcList are arrays of the LocalTensor type.

        
             // The value of NCHW_CONV_ADDR_LIST_SIZE is 16.
template <typename T>
__aicore__ inline void TransDataTo5HD(const LocalTensor<T> (&dstList)[NCHW_CONV_ADDR_LIST_SIZE], const LocalTensor<T> (&srcList)[NCHW_CONV_ADDR_LIST_SIZE], const TransDataTo5HDParams& nchwconvParams)

An array consisting of dstList and srcList of type uint64_t. The array elements correspond to the LocalTensor address values. This API has better performance. You can obtain the address value by calling GetPhyAddr of the LocalTensor.

        
             // The value of NCHW_CONV_ADDR_LIST_SIZE is 16.
template<typename T>
__aicore__ inline void TransDataTo5HD(uint64_t dstList[NCHW_CONV_ADDR_LIST_SIZE], uint64_t srcList[NCHW_CONV_ADDR_LIST_SIZE], const TransDataTo5HDParams& nchwconvParams)

dst and src are LocalTensors of the uint64_t type, which consecutively store the address values of the corresponding LocalTensor. You can obtain the address value by calling GetPhyAddr of the LocalTensor.

       
            template <typename T>
__aicore__ inline void TransDataTo5HD(const LocalTensor<uint64_t>& dst, const LocalTensor<uint64_t>& src, const TransDataTo5HDParams& nchwconvParams)

Parameters

**Table 1** Parameters in the template
Parameter	Description
T	Operand data type. For the Atlas 350 Accelerator Card, the supported data types are int8_t, uint8_t, int16_t, uint16_t, half, bfloat16_t, int32_t, uint32_t, and float. For the Atlas A3 training product / Atlas A3 inference product , the supported data types are int8_t, uint8_t, int16_t, uint16_t, half, int32_t, uint32_t, and float. For the Atlas A2 training product / Atlas A2 inference product , the supported data types are int8_t, uint8_t, int16_t, uint16_t, half, int32_t, uint32_t, and float. For the Atlas 200I/500 A2 inference product , the supported data types are int8_t, uint8_t, int16_t, uint16_t, half, int32_t, uint32_t, and float. For the Atlas inference product AI Core, the supported data types are int8_t, uint8_t, int16_t, uint16_t, half, int32_t, uint32_t, and float. For the Atlas training product , the supported data types are int8_t, uint8_t, int16_t, uint16_t, and half.

**Table 2** Parameters
Parameter	Input/Output	Meaning
dstList	Output	Address sequence of the destination operand. The type is LocalTensor or the address value of the LocalTensor. The TPosition supported by the LocalTensor is VECIN, VECCALC, or VECOUT. The start address of the LocalTensor must be 32-byte aligned. For details about the supported data types, see the description of the template parameter T.
srcList	Input	Address sequence of the source operand. The type is LocalTensor or the address value of the LocalTensor. The TPosition supported by the LocalTensor is VECIN, VECCALC, or VECOUT. The start address of the LocalTensor must be 32-byte aligned. For details about the supported data types, see the description of the template parameter T. Its data type must be the same as that of dstList.
dst	Output	Destination operand. The type is LocalTensor. The address value of the corresponding LocalTensor is stored continuously. The TPosition supported by the LocalTensor is VECIN/VECCALC/VECOUT. The start address of the LocalTensor must be 32-byte aligned.
src	Input	Source operand. The type is LocalTensor. The address value of the corresponding LocalTensor is stored continuously. The TPosition supported by the LocalTensor is VECIN/VECCALC/VECOUT. The start address of the LocalTensor must be 32-byte aligned.
nchwconvParams	Input	Data structure for controlling TransDataTo5HD. The structure contains parameters such as the control parameters of the read and write positions, number of iterations, and address stride between adjacent iterations. For details, see ${INSTALL_DIR}/include/ascendc/basic_api/interface/kernel_struct_transpose.h. Replace ${INSTALL_DIR} with the CANN installation path. For details about the parameter description, see Table 3.

**Table 3** Parameters in the TransDataTo5HDParams structure
Parameter	Input/Output	Description
dstHighHalf	Input	Whether to store the data of dstList to the upper or lower half of the data block. Only the int8_t and uint8_t data types are supported. The supported data type is bool. The options are as follows: True: upper half False: lower half
srcHighHalf	Input	Whether to read the data of srcList from the upper or lower half of the data block. Only the int8_t and uint8_t data types are supported. The supported data type is bool. The options are as follows: True: Data is read from the upper half of a data block. False: Data is read from the lower half of a data block.
repeatTimes	Input	Number of iteration repeats. repeatTimes ∈ [0,255]. For details about this parameter, see High-dimensional Sharding APIs. Notes: When repeatTimes is 1, the effective start position of destination and source operands is computed as the input start position of dstList/srcList plus dstRepStride/srcRepStride. Set dstRepStride/srcRepStride to 0 if you need the effective start position to match the original start position of dstList/srcList. When repeatTimes is greater than 1, the effective start position of the destination and source operands in the first repeat is the input start position of dstList/srcList. The second repeat requires adding dstRepStride/srcRepStride. This rule applies.
dstRepStride	Input	Datablock-to-datablock stride between adjacent repeats of the destination operand, in the unit of data blocks. For details, see repeatStride.
srcRepStride	Input	Datablock-to-datablock stride between adjacent repeats of the source operand, in the unit of data blocks. For details, see repeatStride.

Restrictions

For details about the operand address alignment requirements, see General Address Alignment Restrictions.
For details about the constraints on operand address overlapping, see General Address Overlapping Restrictions.

For conversion from the NCHW format to NC1HWC0, each element in srcList and dstList shall be configured as the starting address of each HW plane.
For optimal performance with 8-bit data types, keep dstHighHalf and srcHighHalf unchanged during HW-direction repeats, then modify them.
All addresses in dst and src must be stored contiguously. For details, see the calling example.

Returns

None

Examples

These examples show only part of the code used in the computation.

Call mode when the input is LocalTensor

        
         
           
           
             AscendC::TransDataTo5HDParams transDataParams;
transDataParams.dstHighHalf = true; // This is valid only for the int8_t or uint8_t inputs. It reads data from the upper half of srcLocalList.
transDataParams.srcHighHalf = true; // This is valid only for the int8_t or uint8_t inputs. It writes data to the upper half of dstLocalList.
transDataParams.repeatTimes = 1; // Number of iteration repeats. Sixteen data blocks are processed in each repeat.
transDataParams.dstRepStride = 0;
transDataParams.srcRepStride = 0;
AscendC::LocalTensor<int8_t> dstLocalList[16];
int width = 32 / sizeof(int8_t); // Number of elements stored in each data block. The value is 32.
for (int i = 0; i < 16; i++) { // dstLocal is a LocalTensor of the int8_t type.
    dstLocalList[i] = dstLocal[width * i];
}
AscendC::LocalTensor<int8_t> srcLocalList[16];
for (int i = 0; i < 16; i++) { // srcLocal is a LocalTensor of the int8_t type.
    srcLocalList[i] = srcLocal[width * i];
}
AscendC::TransDataTo5HD<int8_t>(dstLocalList, srcLocalList, transDataParams);

            

          

        
       

(Recommended) Call mode when the input is LocalTensor address value

        
         
           
           
             AscendC::TransDataTo5HDParams transDataParams;
transDataParams.dstHighHalf = true; // This is valid only for the int8_t or uint8_t inputs. It reads data from the upper half of srcLocalList.
transDataParams.srcHighHalf = true; // This is valid only for the int8_t or uint8_t inputs. It writes data to the upper half of dstLocalList.
transDataParams.repeatTimes = 1; // Number of iteration repeats. Sixteen data blocks are processed in each repeat.
transDataParams.dstRepStride = 0;
transDataParams.srcRepStride = 0;
AscendC::LocalTensor<int8_t> dstLocalList[16];
int width = 32 / sizeof(int8_t); // Number of elements stored in each data block. The value is 32.
uint64_t dstLocalList[16];
for (int i = 0; i < 16; i++) { // dstLocal is a LocalTensor of the int8_t type.
    dstLocalList[i] = (uint64_t)(dstLocal[width * i].GetPhyAddr());
}
uint64_t srcLocalList[16];
for (int i = 0; i < 16; i++) { // srcLocal is a LocalTensor of the int8_t type.
    srcLocalList[i] = (uint64_t)(srcLocal[width * i].GetPhyAddr());
}
AscendC::TransDataTo5HD<int8_t>(dstLocalList, srcLocalList, transDataParams);

            

          

        
       

Call mode when the input is address LocalTensor

        
         
           
           
             AscendC::TransDataTo5HDParams transDataParams;
transDataParams.dstHighHalf = true; // This is valid only for the int8_t or uint8_t inputs. It reads data from the upper half of srcLocalList.
transDataParams.srcHighHalf = true; // This is valid only for the int8_t or uint8_t inputs. It writes data to the upper half of dstLocalList.
transDataParams.repeatTimes = 1; // Number of iteration repeats. Sixteen data blocks are processed in each repeat.
transDataParams.dstRepStride = 0;
transDataParams.srcRepStride = 0;
AscendC::LocalTensor<int8_t> dstLocalList[16];
int width = 32 / sizeof(int8_t); // Number of elements stored in each data block. The value is 32.
// Use TQue to allocate a LocalTensor of uint64_t type to store the addresses of dstLocal and srcLocal.
AscendC::LocalTensor<uint64_t> dst = workQueueSrc1.AllocTensor<uint64_t>();
for (int i = 0; i < 16; i++) { // dstLocal is a LocalTensor of the int8_t type.
    dst.SetValue(i, (uint64_t)(dstLocal[width * i].GetPhyAddr()));
}
AscendC::LocalTensor<uint64_t> src = workQueueSrc2.AllocTensor<uint64_t>();
for (int i = 0; i < 16; i++) { // srcLocal is a LocalTensor of the int8_t type.
    src.SetValue(i, (uint64_t)(srcLocal[width * i].GetPhyAddr()));
}
AscendC::TransDataTo5HD<int8_t>(dst, src, transDataParams);
// Free the LocalTensor address.
workQueueSrc1.FreeTensor(dst);
workQueueSrc2.FreeTensor(src);

            

          

        
       

When the input and output are of the int8_t type, the result is as follows:

Input (src):
[[  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17
   18  19  20  21  22  23  24  25  26  27  28  29  30  31]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49
   50  51  52  53  54  55  56  57  58  59  60  61  62  63]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79  80  81
   82  83  84  85  86  87  88  89  90  91  92  93  94  95]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111 112 113
  114 115 116 117 118 119 120 121 122 123 124 125 126 127]
 [  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17
   18  19  20  21  22  23  24  25  26  27  28  29  30  31]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49
   50  51  52  53  54  55  56  57  58  59  60  61  62  63]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79  80  81
   82  83  84  85  86  87  88  89  90  91  92  93  94  95]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111 112 113
  114 115 116 117 118 119 120 121 122 123 124 125 126 127]
 [  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17
   18  19  20  21  22  23  24  25  26  27  28  29  30  31]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49
   50  51  52  53  54  55  56  57  58  59  60  61  62  63]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79  80  81
   82  83  84  85  86  87  88  89  90  91  92  93  94  95]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111 112 113
  114 115 116 117 118 119 120 121 122 123 124 125 126 127]
 [  0   1   2   3   4   5   6   7   8   9  10  11  12  13  14  15  16  17
   18  19  20  21  22  23  24  25  26  27  28  29  30  31]
 [ 32  33  34  35  36  37  38  39  40  41  42  43  44  45  46  47  48  49
   50  51  52  53  54  55  56  57  58  59  60  61  62  63]
 [ 64  65  66  67  68  69  70  71  72  73  74  75  76  77  78  79  80  81
   82  83  84  85  86  87  88  89  90  91  92  93  94  95]
 [ 96  97  98  99 100 101 102 103 104 105 106 107 108 109 110 111 112 113
  114 115 116 117 118 119 120 121 122 123 124 125 126 127]]
Output (dstGm):
// Read data from the upper half of the input data and write data to the upper half of the output data.
[[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 16 48 80 112 16 48 80 112 16 48 80 112 16 48 80 112 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 17 49 81 113 17 49 81 113 17 49 81 113 17 49 81 113 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 18 50 82 114 18 50 82 114 18 50 82 114 18 50 82 114 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 19 51 83 115 19 51 83 115 19 51 83 115 19 51 83 115 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 20 52 84 116 20 52 84 116 20 52 84 116 20 52 84 116 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 21 53 85 117 21 53 85 117 21 53 85 117 21 53 85 117 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 22 54 86 118 22 54 86 118 22 54 86 118 22 54 86 118 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 23 55 87 119 23 55 87 119 23 55 87 119 23 55 87 119 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 24 56 88 120 24 56 88 120 24 56 88 120 24 56 88 120 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 25 57 89 121 25 57 89 121 25 57 89 121 25 57 89 121 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 26 58 90 122 26 58 90 122 26 58 90 122 26 58 90 122 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 27 59 91 123 27 59 91 123 27 59 91 123 27 59 91 123 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 28 60 92 124 28 60 92 124 28 60 92 124 28 60 92 124 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 29 61 93 125 29 61 93 125 29 61 93 125 29 61 93 125 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 30 62 94 126 30 62 94 126 30 62 94 126 30 62 94 126 ]
[0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 0 31 63 95 127 31 63 95 127 31 63 95 127 31 63 95 127 ]]

When the input and output are of the half type, the result is as follows:

Input (src):
[[  0.   1.   2.   3.   4.   5.   6.   7.   8.   9.  10.  11.  12.  13.
   14.  15.]
 [ 16.  17.  18.  19.  20.  21.  22.  23.  24.  25.  26.  27.  28.  29.
   30.  31.]
 [ 32.  33.  34.  35.  36.  37.  38.  39.  40.  41.  42.  43.  44.  45.
   46.  47.]
 [ 48.  49.  50.  51.  52.  53.  54.  55.  56.  57.  58.  59.  60.  61.
   62.  63.]
 [ 64.  65.  66.  67.  68.  69.  70.  71.  72.  73.  74.  75.  76.  77.
   78.  79.]
 [ 80.  81.  82.  83.  84.  85.  86.  87.  88.  89.  90.  91.  92.  93.
   94.  95.]
 [ 96.  97.  98.  99. 100. 101. 102. 103. 104. 105. 106. 107. 108. 109.
  110. 111.]
 [112. 113. 114. 115. 116. 117. 118. 119. 120. 121. 122. 123. 124. 125.
  126. 127.]
 [128. 129. 130. 131. 132. 133. 134. 135. 136. 137. 138. 139. 140. 141.
  142. 143.]
 [144. 145. 146. 147. 148. 149. 150. 151. 152. 153. 154. 155. 156. 157.
  158. 159.]
 [160. 161. 162. 163. 164. 165. 166. 167. 168. 169. 170. 171. 172. 173.
  174. 175.]
 [176. 177. 178. 179. 180. 181. 182. 183. 184. 185. 186. 187. 188. 189.
  190. 191.]
 [192. 193. 194. 195. 196. 197. 198. 199. 200. 201. 202. 203. 204. 205.
  206. 207.]
 [208. 209. 210. 211. 212. 213. 214. 215. 216. 217. 218. 219. 220. 221.
  222. 223.]
 [224. 225. 226. 227. 228. 229. 230. 231. 232. 233. 234. 235. 236. 237.
  238. 239.]
 [240. 241. 242. 243. 244. 245. 246. 247. 248. 249. 250. 251. 252. 253.
  254. 255.]]

Output (dstGm):
[[  0.  16.  32.  48.  64.  80.  96. 112. 128. 144. 160. 176. 192. 208.
  224. 240.]
 [  1.  17.  33.  49.  65.  81.  97. 113. 129. 145. 161. 177. 193. 209.
  225. 241.]
 [  2.  18.  34.  50.  66.  82.  98. 114. 130. 146. 162. 178. 194. 210.
  226. 242.]
 [  3.  19.  35.  51.  67.  83.  99. 115. 131. 147. 163. 179. 195. 211.
  227. 243.]
 [  4.  20.  36.  52.  68.  84. 100. 116. 132. 148. 164. 180. 196. 212.
  228. 244.]
 [  5.  21.  37.  53.  69.  85. 101. 117. 133. 149. 165. 181. 197. 213.
  229. 245.]
 [  6.  22.  38.  54.  70.  86. 102. 118. 134. 150. 166. 182. 198. 214.
  230. 246.]
 [  7.  23.  39.  55.  71.  87. 103. 119. 135. 151. 167. 183. 199. 215.
  231. 247.]
 [  8.  24.  40.  56.  72.  88. 104. 120. 136. 152. 168. 184. 200. 216.
  232. 248.]
 [  9.  25.  41.  57.  73.  89. 105. 121. 137. 153. 169. 185. 201. 217.
  233. 249.]
 [ 10.  26.  42.  58.  74.  90. 106. 122. 138. 154. 170. 186. 202. 218.
  234. 250.]
 [ 11.  27.  43.  59.  75.  91. 107. 123. 139. 155. 171. 187. 203. 219.
  235. 251.]
 [ 12.  28.  44.  60.  76.  92. 108. 124. 140. 156. 172. 188. 204. 220.
  236. 252.]
 [ 13.  29.  45.  61.  77.  93. 109. 125. 141. 157. 173. 189. 205. 221.
  237. 253.]
 [ 14.  30.  46.  62.  78.  94. 110. 126. 142. 158. 174. 190. 206. 222.
  238. 254.]
 [ 15.  31.  47.  63.  79.  95. 111. 127. 143. 159. 175. 191. 207. 223.
  239. 255.]]

When the input and output are of the int32_t type, the result is as follows:

Input (src):
[[  0   1   2   3   4   5   6   7  ]
 [  8   9  10  11  12  13  14  15  ]
 [ 16  17  18  19  20  21  22  23  ]
 [ 24  25  26  27  28  29  30  31  ]
 [ 32  33  34  35  36  37  38  39  ]
 [ 40  41  42  43  44  45  46  47  ]
 [ 48  49  50  51  52  53  54  55  ]
 [ 56  57  58  59  60  61  62  63  ]
 [ 64  65  66  67  68  69  70  71  ]
 [ 72  73  74  75  76  77  78  79  ]
 [ 80  81  82  83  84  85  86  87  ]
 [ 88  89  90  91  92  93  94  95  ]
 [ 96  97  98  99  100 101 102 103 ]
 [ 104 105 106 107 108 109 110 111 ]
 [ 112 113 114 115 116 117 118 119 ]
 [ 120 121 122 123 124 125  126 127]]
Output data (dstGm):
[[ 0 8 16 24 32 40 48 56 64 72 80 88 96 104 112 120   ]
 [ 1 9 17 25 33 41 49 57 65 73 81 89 97 105 113 121   ] 
 [ 2 10 18 26 34 42 50 58 66 74 82 90 98 106 114 122  ]
 [ 3 11 19 27 35 43 51 59 67 75 83 91 99 107 115 123  ]
 [ 4 12 20 28 36 44 52 60 68 76 84 92 100 108 116 124 ] 
 [ 5 13 21 29 37 45 53 61 69 77 85 93 101 109 117 125 ]
 [ 6 14 22 30 38 46 54 62 70 78 86 94 102 110 118 126 ]
 [ 7 15 23 31 39 47 55 63 71 79 87 95 103 111 119 127 ]]

Parent topic: BlockReduceMin