Documentation
Download

Cast

Applicability

Product

Supported

Atlas 350 Accelerator Card

√

Atlas A3 training product / Atlas A3 inference product

x

Atlas A2 training product / Atlas A2 inference product

x

Atlas 200I/500 A2 inference product

x

Atlas inference product AI Core

x

Atlas inference product Vector Core

x

Atlas training product

x

Function Usage

Converts the data type precision. Converts the source operand of the U type to the destination operand of the T type based on the specified conversion mode.

Prototype

template <typename T = DefaultType, typename U = DefaultType, const CastTrait& trait = castTrait, typename S, typename V>
__simd_callee__ inline void Cast(S& dstReg, V& srcReg, MaskReg& mask)

Parameters

Table 1 Template parameters

Parameter

Description

T

Data type of the destination operand.

U

Data type of the source operand.

trait

Type conversion mode structure.

There are RegLayout, SatMode, MaskMergeMode, and RoundMode.

To enable SatMode, use this API together with the API in SetCtrlSpr(ISASI).

S

srcReg type, for example, RegTensor<float>. It is automatically inferred by the compiler and does not need to be specified.

V

dstReg type, for example, RegTensor<int32_t>. It is automatically inferred by the compiler and does not need to be specified.
Table 2 Parameters

Parameter

Input/Output

Description

dstReg

Output

Destination operand.

The type is RegTensor.

For details about the data types supported by the Atlas 350 Accelerator Card, see Table 3, Table 4, Table 5 and Table 6.

srcReg

Input

Source operand.

The type is RegTensor.

For details about the data types supported by the Atlas 350 Accelerator Card, see Table 3, Table 4, Table 5 and Table 6.

mask

Input

Valid indication of the source operand element operation. For details, see MaskReg.

Note: mask for data type conversion is filtered based on the larger value of sizeof(dtype) in the input and output types.
Table 3 Float to integer

src dtype

dst dtype

mode

round mode

sat mode

layout mode

half

int4x2_t

MaskMergeMode::ZEROING

RoundMode::CAST_RINT,

RoundMode::CAST_ROUND,

RoundMode::CAST_FLOOR,

RoundMode::CAST_CEIL,

RoundMode::CAST_TRUNC

SatMode::NO_SAT

SatMode::SAT

RegLayout::ZERO, RegLayout::ONE, RegLayout::TWO, RegLayout::THREE

float

int16_t

RegLayout::ZERO

RegLayout::ONE

half

int8_t

half

uint8_t

float

int64_t

float

int32_t

RegLayout::UNKNOWN

half

int16_t

half

int32_t

SatMode::UNKNOWN

RegLayout::ZERO

RegLayout::ONE
Table 4 Float to float

src dtype

dst dtype

mode

round mode

sat mode

layout mode

half

float

MaskMergeMode::ZEROING

RoundMode::UNKNOWN

SatMode::UNKNOWN

RegLayout::ZERO

RegLayout::ONE

bfloat16_t

float

hifloat8_t

half

float

bfloat16_t

RoundMode::CAST_RINT,

RoundMode::CAST_ROUND,

RoundMode::CAST_FLOOR,

RoundMode::CAST_CEIL,

RoundMode::CAST_TRUNC

SatMode::NO_SAT

SatMode::SAT

float

half

RoundMode::CAST_ODD,

RoundMode::CAST_RINT,

RoundMode::CAST_ROUND,

RoundMode::CAST_FLOOR

RoundMode::CAST_CEIL,

RoundMode::CAST_TRUNC

half

hifloat8_t

RoundMode::CAST_ROUND,

RoundMode::CAST_HYBRID

float

hifloat8_t

RegLayout::ZERO, RegLayout::ONE, RegLayout::TWO, RegLayout::THREE

float

fp8_e5m2_t

RoundMode::CAST_RINT

float

fp8_e4m3fn_t

hifloat8_t

float

RoundMode::UNKNOWN

SatMode::UNKNOWN

fp8_e4m3fn_t

float

fp8_e5m2_t

float

bfloat16_t

fp4x2_e2m1_t

RoundMode::CAST_RINT,

RoundMode::CAST_ROUND,

RoundMode::CAST_FLOOR,

RoundMode::CAST_CEIL,

RoundMode::CAST_TRUNC

RegLayout::UNKNOWN

bfloat16_t

fp4x2_e1m2_t

fp4x2_e2m1_t

bfloat16_t

RoundMode::UNKNOWN

fp4x2_e1m2_t

bfloat16_t

bfloat16_t

half

RoundMode::CAST_RINT,

RoundMode::CAST_ROUND,

RoundMode::CAST_FLOOR,

RoundMode::CAST_CEIL,

RoundMode::CAST_TRUNC

SatMode::NO_SAT

SatMode::SAT

half

bfloat16_t

SatMode::UNKNOWN

bfloat16_t

fp8_e8m0_t

RoundMode::UNKNOWN

RegLayout::ZERO

RegLayout::ONE

fp8_e8m0_t

bfloat16_t
Table 5 Integer to float

src dtype

dst dtype

mode

round mode

layout mode

int4x2_t

half

MaskMergeMode::ZEROING

RoundMode::UNKNOWN

RegLayout::ZERO, RegLayout::ONE, RegLayout::TWO, RegLayout::THREE

int4x2_t

bfloat16_t

uint8_t

half

RegLayout::ZERO

RegLayout::ONE

int8_t

half

int16_t

float

int64_t

float

RoundMode::CAST_RINT,

RoundMode::CAST_ROUND,

RoundMode::CAST_FLOOR,

RoundMode::CAST_CEIL,

RoundMode::CAST_TRUNC

int16_t

half

RegLayout::UNKNOWN

int32_t

float
Table 6 Integer to integer

src dtype

dst dtype

mode

sat mode

layout mode

uint32_t

uint8_t

MaskMergeMode::ZEROING

SatMode::NO_SAT

SatMode::SAT

RegLayout::ZERO, RegLayout::ONE, RegLayout::TWO, RegLayout::THREE

int32_t

uint8_t

uint16_t

uint8_t

RegLayout::ZERO

RegLayout::ONE

int16_t

uint8_t

uint32_t

uint16_t

uint32_t

int16_t

int32_t

uint16_t

int32_t

int16_t

uint8_t

uint16_t

RoundMode::UNKNOWN

int8_t

int16_t

uint16_t

uint32_t

int16_t

uint32_t

int16_t

int32_t

int32_t

int64_t

uint8_t

uint32_t

RegLayout::ZERO, RegLayout::ONE, RegLayout::TWO, RegLayout::THREE

int8_t

int32_t

int16_t

int4x2_t

int4x2_t

int16_t

int64_t

int32_t

SatMode::NO_SAT

SatMode::SAT

RegLayout::ZERO

RegLayout::ONE
Table 7 Float-to-integer conversion rules

Conversion Mode

Conversion Rule

CAST_RINT

Rounding to the nearest even number when a number is halfway between two others.

Examples:

Input: 3.3; output: 3

Input: 5.9; output: 6

Input: 5.5; output 6. Because 6 is an even number and 5 is an odd number, the value is rounded up to 6.

Input: 4.5; output: 4 (4 is an even number)

Input: –2.4; output: 2

Input: –3.6; output: –4

CAST_ROUND

Rounding away from zero when a number is halfway between two others.

Examples:

Input: 3.3; output: 3

Input: 5.9; output: 6

Input: 5.5; output: 6. Because 6 is farther from 0 than 5, the value is rounded up to 6.

Input: –2.4; output: 2

Input: –3.6; output: –4

Input: –6.5; output: –7. Because –7 is farther from 0 than –6, the value is rounded down to –7.

CAST_FLOOR

Rounding toward negative infinity.

Examples:

Input: 3.2; output: 3

Input: 7.9; output: 7

Input: –4.6; output: –5

Input: –3.1; output: –4

The output is closer to negative infinity than the input.

CAST_CEIL

Rounding toward positive infinity.

Examples:

Input: 3.2; output: 4

Input: 7.9; output: 8

Input: –4.6; output: –4

Input: –3.1; output: –3

The output is closer to positive infinity than the input.

CAST_TRUNC

Rounding toward zero.

Examples:

Input: 3.2; output: 3

Input: 7.9; output: 7

Input: –4.6; output: –4

Input: –3.1; output: –3

The output is closer to 0 than the input.
Table 8 Float-to-float conversion rules

src dtype

dst dtype

Rounding Rule

float

half

CAST_ODD: rounding to the nearest odd number.

The half has 10 bits of mantissa, and float has 23 bits of mantissa. When converting float to half, 13 bits are discarded. The least significant bit of float16 is marked in yellow y, and the discarded precision of float is marked in red.

float +/-a.xxxxxxxxx00000100000000: If the bit in y is 0 (even number), rounding is performed, and half a.xxxxxxxxx1 is obtained.

float +/-a.xxxxxxxxx11000100000000: If the bit in y is 1 (odd number), no rounding is performed, and half a.xxxxxxxxx1 is obtained.

Example:

Input: float 123.23333, represented in binary as follows:

sign:0, exponent:10000101, mantissa:11101100111011101110111

float_exponent = 0b10000101 = 133

half_exponent - 15 = 133 - 127

half_exponent = 21 = 0b10101

The 14th bit of float_mantissa is 1 (odd number), so no rounding is performed and the last 13 bits are discarded.

11101100111011101110111

half_mantissa = 1110110011 obtained

Therefore, the binary value of half is:

sign:0, exponent:10101, mantissa:1110110011

half = 123.2

float

bfloat16_t

The binary rule of float is as follows:

sign:1bit,exponent:8bit mantissa:23bit

The binary rule of bfloat16_t is as follows:

sign:1bit,exponent:8bit,mantissa:7bit

CAST_RINT: rounding to the nearest even number when a number is halfway between two others.

Examples:

1. float32_mantissa:00100000010000010000000

If the discarded precision is less than 0b1000000000000000, which is equivalent to [0, 0.5), no rounding is performed.

bfloat16_mantissa:0010000 is obtained.

2. float32_mantissa:00110001000000000000000

If the discarded precision is equal to 0b1000000000000000 and the least significant bit of bf16 is an even number, no rounding is performed.

bfloat16_mantissa:0011000 is obtained.

3. float32_mantissa:00110011000000000000000

If the discarded precision is equal to 0b1000000000000000 and the least significant bit of bf16 is an odd number, rounding is performed.

bfloat16_mantissa:0011010 is obtained.

4. float32_mantissa:00100001010000010000000

If the discarded precision is greater than 0b1000000000000000, which is equivalent to (0.5, 1), rounding is performed.

bfloat16_mantissa:0010001 is obtained.

CAST_ROUND: rounding away from zero when a number is halfway between two others.

Examples:

1. float32_mantissa:00100000010000010000000

If the discarded precision is less than 0b1000000000000000, which is equivalent to [0, 0.5), no rounding is performed.

bfloat16_mantissa:0010000 is obtained.

2. float32_mantissa:00110001000000000000000

The discarded precision is equal to 0b1000000000000000. After rounding, the value is farther away from 0. Therefore, rounding is performed.

bfloat16_mantissa:0011001 is obtained.

3. float32_mantissa:00100001010000010000000

If the discarded precision is greater than 0b1000000000000000, which is equivalent to (0.5, 1), rounding is performed.

bfloat16_mantissa:0010001 is obtained.

CAST_FLOOR: rounding toward negative infinity.

1. float32_mantissa:00100000010000010000000: If the float32 value is positive, no rounding is performed, and bfloat16_mantissa:0010000 is obtained.

2. float32_mantissa:00110001000000000000000

If the float32 value is negative, rounding is performed, and bfloat16_mantissa:0011001 is obtained.

CAST_CEIL: rounding toward positive infinity.

1. float32_mantissa:00100000010000010000000: If the float32 value is positive, rounding is performed, and bfloat16_mantissa:0010001 is obtained.

2. float32_mantissa:00110001000000000000000

If the float32 value is negative, no rounding is performed, and bfloat16_mantissa:0011000 is obtained.

CAST_TRUNC: rounding toward zero.

1. float32_mantissa:00100000010000010000000: No rounding is performed, and the red precision is directly discarded. The result is bfloat16_mantissa:001000.

Complete example: Input the float32 value 205.75, represented in binary as:

sign:0, exponent:10000110, mantissa:10011011100000000000000

CAST_RINT

float32_mantissa = 0b10011011100000000000000

If the discarded precision is greater than 0b1000000000000000, which is equivalent to (0.5, 1), rounding is performed.

bfloat16_mantissa = 0b1001110 is obtained.

Therefore, the binary value of bfloat16 is as follows:

sign:0, exponent:10000110, mantissa:1001110

bfloat16 = 206

bfloat16_t

half

The binary rule of bfloat16_t is as follows:

sign:1bit,exponent:8bit,mantissa:7bit

The binary rule of float16 is as follows:

sign:1bit,exponent:5bit,mantissa:10bit

The mantissa bit difference is 3 bits.

When bfloat16_exponent = exponent – 127 ≥ –14:

Since bfloat16 has lower precision and float16 has higher precision, converting from bfloat16 to float16 does not reduce precision.

Example 1: bfloat16_mantissa = 0b1011000 ->

float16_maintissa = 0b1011000000. No rounding is performed on the binary bits, and only three 0s need to be added to the lower bits.

When bfloat16_exponent = exponent – 127 < –14:

Example 2: When exponent is 108 and bfloat16_exponent is exponent – 127 = –19 < –14, the exponent must be increased by 5 to be equal to float16_exponent because the minimum value of float16_exponent (exponent – 14) is –14 and cannot represent –19. In addition, the mantissa needs to be divided by 2 to the power of 5 because the bf16 computation formula is as follows: bf16 = s × (2^(e – 127)) × (man) = s × (2^(e – 127 + 5)) × (man * 2^(–5)). By multiplying the exponent and dividing the mantissa by the same factor, the final value remains unchanged. Assume that bfloat16_mantissa = 0b1011011.

The bfloat16 mantissa is computed as follows: 1 + man/128. The binary representation is as follows:

0b1.1011011. When the exponent bit is multiplied by 2 to the power of 5, the mantissa must be divided by 2 to the power of 5, and the decimal point is moved five places to the left. Therefore, bfloat16_mantissa is 0b0.000011011011. The mantissa has 12 bits. Because float16 has only 10-bit mantissa, the two least significant bits need to be discarded in the following CAST modes, and whether to round is determined based on the rounding mode. In the current example, the discarded precision midpoint is 0b10.

CAST_RINT: rounding to the nearest even number when a number is halfway between two others.

When bfloat16_mantissa is less than or equal to 10 bits, rounding is not required.

When bfloat16_mantissa is greater than 10 bits, examples similar to example 2 are as follows:

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: bfloat16_mantissa = 0b0.00001101100xxx

0xxx < 1000. Output: float16_mantissa = 0b0.0000110110

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: bfloat16_mantissa = 0b0.00001101100xxx

1x1x > 1000. Output: float16_mantissa = 0b0.0000110111

(3) When the discarded precision is equal to the discarded precision midpoint and the 10th bit is an even number, no rounding is performed.

Example: bfloat16_mantissa = 0b0.00001101100xxx

0 % 2 == 0. Output: float16_mantissa = 0b0.0000110110

(4) When the discarded precision is equal to the discarded precision midpoint and the 10th bit is an odd number, rounding is performed.

Example: bfloat16_mantissa = 0b0.00001100010xxx

1 % 2 == 1. Output: float16_mantissa = 0b0.0000110001

CAST_ROUND: rounding away from zero when a number is halfway between two others.

When bfloat16_mantissa is less than or equal to 10 bits, rounding is not required.

When bfloat16_mantissa is greater than 10 bits, examples similar to example 2 are as follows:

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: bfloat16_mantissa = 0b0.00001101100xxx

0xxx < 1000. Output: float16_mantissa = 0b0.0000110110

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: bfloat16_mantissa = 0b0.00001101100xxx

1x1x > 1000. Output: float16_mantissa = 0b0.0000110111

(3) When the discarded precision is equal to the discarded precision midpoint, rounding away from zero is used. Therefore, rounding is performed.

Example: bfloat16_mantissa = 0b0.00001101100xxx

Output: float16_mantissa = 0b0.0000110111

CAST_FLOOR: rounding toward negative infinity.

When bfloat16_mantissa is less than or equal to 10 bits, rounding is not required.

When bfloat16_mantissa is greater than 10 bits, examples similar to example 2 are as follows:

(1) When the sign bit S is 0, rounding is not performed for positive numbers.

Example: bfloat16_mantissa = 0b0.00001101100xxx

Output: float16_mantissa = 0b0.0000110110

(2) When the sign bit S is 1, rounding is performed for negative numbers.

Example: bfloat16_mantissa = 0b0.00001101100xxx

Output: float16_mantissa = 0b0.0000110111

CAST_CEIL: rounding toward positive infinity.

When bfloat16_mantissa is less than or equal to 10 bits, rounding is not required.

When bfloat16_mantissa is greater than 10 bits, examples similar to example 2 are as follows:

(1) When the sign bit S is 0, rounding is performed for positive numbers.

Example: bfloat16_mantissa = 0b0.00001101100xxx

Output: float16_mantissa = 0b0.0000110111

(2) When the sign bit S is 1, rounding is not performed for negative numbers.

Example: bfloat16_mantissa = 0b0.00001101100xxx

Output: float16_mantissa = 0b0.0000110110

CAST_TRUNC: rounding toward zero.

When bfloat16_mantissa is less than or equal to 10 bits, rounding is not required.

When bfloat16_mantissa is greater than 10 bits, examples similar to example 2 are as follows:

(1) The extra precision is directly discarded, and no rounding is performed.

Example: bfloat16_mantissa = 0b0.00001101100xxx

Output: float16_mantissa = 0b0.0000110110

Complete example:

Input the bfloat16 value 2.90573e-06, represented in binary as follows:

sign:0, exponent:01101100, mantissa:1000011

bfloat16_exponent = 108 – 127 = -19 < –14. The minimum value of float16_exponent is –14. Therefore, bfloat16_exponent is incremented by 5, and the decimal point of bfloat16_mantissa is shifted leftward.

5 digits, 1.1000011 -> 0.000011000011

CAST_RINT

The discarded precision 11 is greater than the discarded precision midpoint 10. Therefore, rounding is performed.

bfloat16_mantissa = 0.0000110001

The final binary value of float16 is as follows:

sign:0, exponent: 00000, mantissa = 0000110001

The decimal value computed using the formula is as follows:

float16=0.00000293

half

bfloat16_t

The binary rule of half is as follows:

sign:1bit,exponent:5bit,mantissa:10bit

The binary rule of bfloat16_t is as follows:

sign:1bit,exponent:8bit,mantissa:7bit

CAST_RINT: rounding to the nearest even number when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint 100, no rounding is performed.

Example: half_mantissa = 0111010011

Output: bfloat16_mantissa = 0111010

(2) When the discarded precision is greater than the discarded precision midpoint 100, rounding is performed.

Example: half_mantissa = 0111010111

Output: bfloat16_mantissa = 0111011

(3) When the rounding precision is equal to the discarded precision midpoint 100, the least significant bit of bfloat16 is an even number, and no rounding is performed.

Example: half_mantissa = 0111010100

Output: bfloat16_mantissa = 0111010

(4) When the rounding precision is equal to the discarded precision midpoint 100, the least significant bit of bfloat16 is an odd number, and rounding is performed.

Example: half_mantissa = 0111010100

Output: bfloat16_mantissa = 0111011

CAST_ROUND: rounding to the nearest integer. Rounding away from zero when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint 100, no rounding is performed.

Example: half_mantissa = 0111010011

Output: bfloat16_mantissa = 0111010

(2) When the discarded precision is greater than the discarded precision midpoint 100, rounding is performed.

Example: half_mantissa = 0111010111

Output: bfloat16_mantissa = 0111011

(3) When the rounding precision is equal to the discarded precision midpoint 100, rounding away from zero is used. There, rounding is performed.

Example: half_mantissa = 0111010100

Output: bfloat16_mantissa = 0111011

CAST_FLOOR: rounding toward negative infinity.

(1) When the input is positive, no rounding is performed.

Example: half_mantissa = 0111010011

Output: bfloat16_mantissa = 0111010

(2) When the input is negative, rounding is performed.

Example: half_mantissa = 0111010011

Output: bfloat16_mantissa = 0111011

CAST_CEIL: rounding toward positive infinity.

(1) When the input is positive, rounding is performed.

Example: half_mantissa = 0111010011

Output: bfloat16_mantissa = 0111011

(2) When the input is negative, no rounding is performed.

Example: half_mantissa = 0111010011

Output: bfloat16_mantissa = 0111010

CAST_TRUNC: rounding toward zero.

The extra precision is directly discarded.

Example: half_mantissa = 0111010011

Output: bfloat16_mantissa = 0111010

Complete example:

Input: half 0.131

Binary value: sign:0, exponent:01100, mantissa:0000110001

half_exponent = 12 – 15 = bfloat16_exp – 127

bfloat16_exp = 124 = 0b1111100

In CAST_ROUND mode:

half_mantissa = 0000110001

The discarded precision is less than the discarded precision midpoint 100, and no rounding is performed. Therefore, the result is as follows:

bfloat16_mantissa = 0000110

Therefore, the binary value of bfloat16 is as follows:

sign:0, exponent:1111100, mantissa: 0000110

The decimal value computed using the formula is as follows:

bfloat16 = 0.130859

float

fp8_e4m3fn_t

The binary rule of float is as follows:

sign:1bit,exponent:8bit mantissa:23bit

The binary rule of fp8_e4m3fn_t is as follows:

sign:1bit,exponent:4bit,mantissa:3bit

The 20 bits need to be discarded, and the discarded precision midpoint is 0b10000000000000000000.

CAST_RINT: rounding to the nearest even number when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: float_mantissa = 01100100110000000000000

Output: fp8_e4m3fn_t_mantissa = 011

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: float_mantissa = 01110100110000000000000

Output: fp8_e4m3fn_t_mantissa = 100

(3) When the rounding precision is equal to the discarded precision midpoint, the least significant bit of fp8_e4m3fn_t is an even number, and no rounding is performed.

Example: float_mantissa = 01010000000000000000000

Output: fp8_e4m3fn_t_mantissa = 010

(4) When the rounding precision is equal to the discarded precision midpoint, the least significant bit of fp8_e4m3fn_t is an odd number, and rounding is performed.

Example: float_mantissa = 01110000000000000000000

Output: fp8_e4m3fn_t_mantissa = 100

Complete example:

Input: float 1.233

Binary:

sign:0, exponent:01111111, mantissa:00111011101001011110010

float_exponent = 127-127 = fp8_e4m3fn_t_exp - 7

fp8_e4m3fn_t_exp = 7 = 0b0111

CAST_RINT

float_mantissa = 00111011101001011110010

The discarded precision is greater than the discarded precision midpoint, and rounding is performed. Therefore, the result is as follows:

fp8_e4m3fn_t_mantissa = 010

Therefore, the binary value of fp8_e4m3fn_t is as follows:

sign:0, exponent:0111, mantissa: 010

The decimal value computed using the formula is as follows:

fp8_e4m3fn_t=1.25

float

fp8_e5m2_t

The binary rule of float is as follows:

sign:1bit,exponent:8bit mantissa:23bit

The binary rule of fp8_e5m2_t is as follows:

sign:1bit,exponent:5bit,mantissa:2bit

The 21 bits need to be discarded, and the discarded precision midpoint is 0b100000000000000000000.

CAST_RINT: rounding to the nearest even number when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: float_mantissa = 01000100110000000000000

Output: fp8_e5m2_t_mantissa = 01

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: float_mantissa = 01101001100000000000000

Output: fp8_e5m2_t_mantissa = 10

(3) When the rounding precision is equal to the discarded precision midpoint, the least significant bit of fp8_e5m2_t is an even number, and no rounding is performed.

Example: float_mantissa = 00100000000000000000000

Output: fp8_e5m2_t_mantissa = 00

When the rounding precision is equal to the discarded precision midpoint, the least significant bit of fp8_e5m2_t is an odd number, and rounding is performed.

Example: float_mantissa = 0110000000000000000000

Output: fp8_e5m2_t_mantissa = 10

Complete example:

Input: float32 1.233

Binary:

sign:0, exponent:01111111, mantissa:00111011101001011110010

float_exponent = 127-127 = fp8_e5m2_t_exp - 15

fp8_e5m2_t_exp = 15 = 0b01111

CAST_RINT

float_mantissa = 00111011101001011110010

The discarded precision is greater than the discarded precision midpoint, and rounding is performed. Therefore, the result is as follows:

fp8_e5m2_t_mantissa = 01

Therefore, the binary format of fp8_e5m2_t is as follows:

sign:0, exponent:01111, mantissa: 01

The decimal value computed using the formula is as follows:

fp8_e5m2_t=1.25

bfloat16_t

fp4x2_e2m1_t

The binary rule of bfloat16 is as follows:

sign:1bit,exponent:8bit,mantissa:7bit

The binary rule of fp4x2_e2m1_t is as follows:

sign:1bit,exponent:2bit,mantissa:1bit

The mantissa difference is 6 bits.

When bfloat16_exponent = exponent – 127 < 0:

Example 1: When exponent is 124 and bfloat16_exponent is exponent – 127 = –3 < 0, the exponent must be increased by 3 to be equal to float4_e2m1_exponent because the minimum value of float4_e2m1_exponent (exponent – 1) is 0 and cannot represent –3. In addition, the mantissa needs to be divided by 2 to the power of 3 because the bf16 computation formula is as follows: s × (2^(e – 127)) × (man) = s × (2^(e – 127 + 3)) × (man * 2^(–3)). By multiplying the exponent and dividing the mantissa by the same factor, the final value remains unchanged. Assume that bfloat16_mantissa = 0b1011011.

The bfloat16 mantissa is computed as follows: 1 + man/128. The binary representation is as follows:

0b1.1011011. When the exponent bit is multiplied by 2 to the power of 3, the mantissa must be divided by 2 to the power of 3, and the decimal point is moved three places to the left. Therefore, bfloat16_mantissa is 0b0.0011011011. Because float4_e2m1 has only one mantissa bit, the nine least significant bits need to be discarded in the following CAST modes, and whether to round is determined based on the rounding mode. In the current example, the discarded precision midpoint is 0b100000000.

CAST_RINT: rounding to the nearest even number when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0000110xxx

000110xxx < Discarded precision midpoint. Output: float4_e2m1_mantissa = 0b0.0

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: bfloat16_mantissa = 0b0.0100110xxx

100110xxx > Discarded precision midpoint. Output: float4_e2m1_mantissa = 0b0.1

(3) When the discarded precision is equal to the discarded precision midpoint and the first bit is an even number, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0100000xxx

0 % 2 == 0. Output: float4_e2m1_mantissa = 0b0.0

(4) When the discarded precision is equal to the discarded precision midpoint and the first bit is an odd number, rounding is performed.

Example: bfloat16_mantissa = 0b0.1100000xxx

1 % 2 == 1. Output: float4_e2m1_mantissa = 0b0.1

CAST_ROUND: rounding away from zero when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0000110xxx

000110xxx < Discarded precision midpoint. Output: float4_e2m1_mantissa = 0b0.0

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: bfloat16_mantissa = 0b0.0100110xxx

100110xxx > Discarded precision midpoint. Output: float4_e2m1_mantissa = 0b0.1

(3) When the discarded precision is equal to the discarded precision midpoint, rounding away from zero is used. Therefore, rounding is performed.

Example: bfloat16_mantissa = 0b0.0100000xxx

Output: float4_e2m1_mantissa = 0b0.1

CAST_FLOOR: rounding toward negative infinity.

(1) If the input is positive, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0000110xxx

Output: float4_e2m1_mantissa = 0b0.0

(2) If the input is negative, rounding is performed.

Example: bfloat16_mantissa = 0b0.0100110xxx

Output: float4_e2m1_mantissa = 0b0.1

CAST_CEIL: rounding toward positive infinity.

(1) If the input is positive, rounding is performed.

Example: bfloat16_mantissa = 0b0.0000110xxx

Output: float4_e2m1_mantissa = 0b0.1

(2) If the input is negative, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0100110xxx

Output: float4_e2m1_mantissa = 0b0.0

CAST_TRUNC: rounding toward zero.

(1) The extra precision is directly discarded, and no rounding is performed.

Example: bfloat16_mantissa = 0b0.0100110xxx

Output: float4_e2m1_mantissa = 0b0.0

Complete example:

Input the bfloat16 value 0.761719, represented in binary as follows:

sign:0, exponent:01111110, mantissa:1000011

bfloat16_exponent = 126 – 127 = –1 < 0. The minimum value of float16_exponent is 0. Therefore, bfloat16_exponent is incremented by 1, and the decimal point of bfloat16_mantissa is shifted leftward.

1 digit, 1.1000011 -> 0.11000011

CAST_RINT

The discarded precision 1000011 is greater than the discarded precision midpoint 1000000. Therefore, rounding is performed.

float4_e2m1_mantissa = 0.1

The final binary value of float4_e2m1 is as follows:

sign:0, exponent: 00, mantissa = 1

The decimal value computed using the formula is as follows:

float4_e2m1=0.5

bfloat16_t

fp4x2_e1m2_t

The binary rule of bfloat16 is as follows:

sign:1bit,exponent:8bit,mantissa:7bit

The binary rule of fp4x2_e1m2_t is as follows:

sign:1bit,exponent:1bit,mantissa:2bit

The mantissa bit difference is 5 bits.

When bfloat16_exponent = exponent – 127 < 0, bfloat16_mantissa needs to be shifted.

Example 1: When exponent is 124 and bfloat16_exponent is exponent – 127 = –3 < 0, the exponent must be increased by 3 to be equal to float4_e2m1_exponent because the minimum value of float4_e1m2_exponent (exponent – 1) is 0 and cannot represent –3. In addition, the mantissa needs to be divided by 2 to the power of 3 because the bf16 computation formula is as follows: s × (2^(e – 127)) × (man) = s × (2^(e – 127 + 3)) × (man * 2^(–3)). By multiplying the exponent and dividing the mantissa by the same factor, the final value remains unchanged. Assume that bfloat16_mantissa = 0b1011011.

The bfloat16 mantissa is computed as follows: 1 + man/128. The binary representation is as follows:

0b1.1011011. When the exponent bit is multiplied by 2 to the power of 3, the mantissa must be divided by 2 to the power of 3, and the decimal point is moved three places to the left. Therefore, bfloat16_mantissa is 0b0.0011011011. Because float4_e2m1 has only 2-bit mantissa, the eight least significant bits need to be discarded in the following CAST modes, and whether to round is determined based on the rounding mode. In the current example, the discarded precision midpoint is 0b10000000.

CAST_RINT: rounding to the nearest even number when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0000011xxx

00011xxx < Discarded precision midpoint. Output: float4_e1m2_mantissa = 0b0.00

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: bfloat16_mantissa = 0b0.0010011xxx

10011xxx > Discarded precision midpoint. Output: float4_e1m2_mantissa = 0b0.01

(3) When the discarded precision is equal to the discarded precision midpoint and the second digit is an even number, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0010000xxx

0 % 2 == 0. Output: float4_e1m2_mantissa = 0b0.00

(4) If the discarded precision is equal to the discarded precision midpoint and the second digit is an odd number, rounding is performed.

Example: bfloat16_mantissa = 0b0.0110000xxx

1 % 2 == 1. Output: float4_e1m2_mantissa = 0b0.10

CAST_ROUND: rounding away from zero when a number is halfway between two others.

(1) When the discarded precision is less than the discarded precision midpoint, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0000110xxx

00110xxx < Discarded precision midpoint. Output: float4_e1m2_mantissa = 0b0.00

(2) When the discarded precision is greater than the discarded precision midpoint, rounding is performed.

Example: bfloat16_mantissa = 0b0.0010110xxx

10110xxx > Discarded precision midpoint. Output: float4_e1m2_mantissa = 0b0.01

(3) When the discarded precision is equal to the discarded precision midpoint, rounding away from zero is used. Therefore, rounding is performed.

Example: bfloat16_mantissa = 0b0.0010000xxx

Output: float4_e1m2_mantissa = 0b0.01

CAST_FLOOR: rounding toward negative infinity.

(1) If the input is positive, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0000110xxx

Output: float4_e1m2_mantissa = 0b0.00

(2) If the input is negative, rounding is performed.

Example: bfloat16_mantissa = 0b0.00100110xxx

Output: float4_e1m2_mantissa = 0b0.01

CAST_CEIL: rounding toward positive infinity.

(1) If the input is positive, rounding is performed.

Example: bfloat16_mantissa = 0b0.0000110xxx

Output: float4_e1m2_mantissa = 0b0.01

(2) If the input is negative, no rounding is performed.

Example: bfloat16_mantissa = 0b0.0010110xxx

Output: float4_e1m2_mantissa = 0b0.00

CAST_TRUNC: rounding toward zero.

(1) The extra precision is directly discarded, and no rounding is performed.

Example: bfloat16_mantissa = 0b0.0010110xxx

Output: float4_e1m2_mantissa = 0b0.00

Complete example:

Input the bfloat16 value 0.761719, represented in binary as follows:

sign:0, exponent:01111110, mantissa:1000011

bfloat16_exponent = 126 – 127 = –1 < 0. The minimum value of float16_exponent is 0. Therefore, bfloat16_exponent is incremented by 1, and the decimal point of bfloat16_mantissa is shifted leftward.

1 digit, 1.1000011 -> 0.11000011

CAST_RINT

The discarded precision 000011 is less than the discarded precision midpoint 100000. Therefore, no rounding is performed.

float4_e2m1_mantissa = 0.11

The final binary value of float4_e2m1 is as follows:

sign:0, exponent: 0, mantissa = 11

The decimal value computed using the formula is as follows:

float4_e2m1=0.75

float

hifloat8_t

See Table 9.
Table 9 Conversion rules of float to hifloat8_t

VCVTFF:F322HiF8

tmp2[31 : 0] = f32_src_data[i][31 : 0];

Ev = tmp2[30 : 23] - 8'b01111111, thr = tmp2[13 : 0];

if (Ev == 128 && tmp2[22 : 0] != 23'b0) then

tmp3[7 : 0] = HiF8NAN(8'b10000000);

else if ((Ev == 128 && tmp2[22 : 0] == 23'b0) || (Ev > 15)) then

tmp3[7 : 0] = (tmp2[31] == 1'b0) ? HiF8+INF (8'b01101111) : HiF8-INF (8'b11101111);

else if (Ev < -23) then

tmp3[7 : 0] = HiF8ZERO (8'b00000000);

else if (Ev == -23) then

if ((Half To Away Round) || (Hybrid Round && {1'b1, tmp2[22 : 10]} >= thr)) then

tmp3[7 : 0] = (tmp2[31] == 1'b0) ? 8'b00000001 : 8'b10000001; // min subnormal

else

tmp3[7 : 0] = HiF8ZERO (8'b00000000);

end if

else

if (Ev == 0) then

M = tmp2[22 : 20], T A_bit = tmp2[19], frac = tmp2[19 : 6];

else if (Ev == ±1) then

M = tmp2[22 : 20], T A_bit = tmp2[19], frac = tmp2[19 : 6];

else if (Ev == ±[2, 3]) then

M = tmp2[22 : 20], T A_bit = tmp2[19], frac = tmp2[19:6];

else if (Ev == ±[4, 7]) then

M = tmp2[22 : 21], T A_bit = tmp2[20], frac = tmp2[20 : 7];

else if (Ev == ±[8, 15]) then

M = tmp2[22], T A_bit = tmp2[21], frac = tmp2[21 : 8];

else if (Ev == [-16, -22]) then

M = Ev + 23, T A_bit = tmp2[22], frac = tmp2[22 : 9]; // subnormal

end if

if (Ev == ±[0, 3]) then

if (T A_bit == 1'b1) then

M_tmp = M + 1, Ev = Ev + carry of M_tmp, M = (carry of M_tmp) ? 0 : M_tmp;

end if

else if (Ev == ±[4, 15]) then

if (HALF To Away Round && T A_bit == 1'b1) || (Hybrid Round && frac >= thr) then

M_tmp = M + 1, Ev = Ev + carry of M_tmp, M = (carry of M_tmp) ? 0 : M_tmp;

end if

else if (Ev == [-16, -22]) then

if ((Half To Away Round) && T A_bit == 1'b1) || ((Hybrid Round && frac >= thr) then

M_tmp = Ev + 23, Ev = Ev + 1, M = (Ev == -15) ? 0 : M_tmp;

end if

end if

encode {tmp3[31], Ev, M} to tmp3[7 : 0] following HiF encoding rule;

end if

result[i][7 : 0] = saturation(tmp3[7 : 0]) according to control bit;
Table 10 Conversion rules of half to hifloat8_t

VCVTFF: F162HiF8

tmp2[15 : 0] = f16_src_data[i][15 : 0];

Ev = tmp2[14 : 10] - 5'b01111, thr = {tmp2[0], 1'b1};

if (Ev == 16 && tmp2[9 : 0] != 10'b0) then

tmp3[7 : 0] = HiF8NAN(8'b10000000);

else if (Ev == 16 && tmp2[9:0] == 10'b0) && (Ev > 15) then

tmp3[7 : 0] = (tmp2[31] == 1'b0) ? HiF8+INF(8'b01101111) : HiF8-INF(8'b11101111);

else if (Ev < -23) then

tmp3[7 : 0] = HiF8ZERO(8'b00000000);

else if (Ev == -23) then

if ((Half To Away Round) || (Hybrid Round && {1'b1, tmp2[9]} >= thr)) then

tmp3[7 : 0] = (tmp2[31] == 1'b0) ? 8'b00000001 : 8'b10000001; // min subnormal

else

tmp3[7 : 0] = HiF8ZERO(8'b00000000);

end if

else

if (Ev == 0) then

M = tmp2[9 : 7], T A_bit = tmp2[6], frac = tmp2[6 : 5];

else if (Ev == ±1) then

M = tmp2[9 : 7], T A_bit = tmp2[6], frac = tmp2[6 : 5];

else if (Ev == ±[2, 3]) then

M = tmp2[9 : 7], T A_bit = tmp2[6], frac = tmp2[6 : 5];

else if (Ev == ±[4, 7]) then

M = tmp2[9 : 8], T A_bit = tmp2[7], frac = tmp2[7 : 6];

else if (Ev == ±[8, 15]) then

M = tmp2[9], T A_bit = tmp2[8], frac = tmp2[8 : 7];

else if (Ev == [-16, -22]) then

M = Ev + 23, T A_bit = tmp2[9], frac = tmp2[9 : 8]; // subnormal

end if

if (Ev == ±[0, 3]) then

if (T A_bit == 1'b1) then

M_tmp = M + 1, Ev = Ev + carry of M_tmp, M = (carry of M_tmp) ? 0 : M_tmp;

end if

else if (Ev == ±[4, 15]) then

if (HALF To Away Round && T A_bit == 1'b1) || (Hybrid Round && frac >= thr) then

M_tmp = M + 1, Ev = Ev + carry of M_tmp, M = (carry of M_tmp) ? 0 : M_tmp;

end if

else if (Ev == [-16, -22]) then

if ((Half To Away Round) && T A_bit == 1'b1) || ((Hybrid Round && frac >= thr) then

M_tmp = Ev + 23, Ev = Ev + 1, M = (Ev == -15) ? 0 : M_tmp;

end if

end if

encode {tmp3[31], Ev, M} to tmp3[7 : 0] following HiF encoding rule;

end if

result[i][7 : 0] = saturation(tmp3[7 : 0]) according to control bit;

Returns

None

Constraints

  • This instruction must be used together with the SetCtrlSpr(ISASI) instruction (SetCtrlSpr(ISASI)). You can set the register value to control the saturation and non-saturation modes of Cast.
  • Float to integer:
    • When the input is out of range of the output type, the result is truncated to fit the target data width. For example, if the input is the half-precision value 257, the output will be 1 when cast to uint8_t. If the input is +/–inf, the output will be the corresponding maximum or minimum value of the output type. If the input is nan, the output will be 0.
    • Saturation mode: When the input is out of range of the output type, the output will be the maximum or minimum value of the output type. For example, if the input is the half-precision value 257, the output will be 255 when cast to uint8_t. If the input is the half-precision value –inf, the output will be 0 when cast to uint8_t. If the input is nan, the output will be 0.
  • Float to float:
    • Currently, float-to-float conversion supports both the saturation and non-saturation modes. In non-saturation mode, if the input is nan, the output will be nan. If the input is +/–inf, the output will be +/-inf. In saturation mode, if the input is nan, the output will be 0. If the input is out of range of the output type, the output will be the maximum value of the output type.
    • When the output type is float32, only the non-saturation mode is supported.
    • When the output type is fp8_e4m3fn_t, the output is nan because fp8_e4m3fn_t does not have the inf representation format.
    • When the output type is fp8_e5m2_t/fp8_e4m3fn_t and the input is nan, the default output is 0.
    • For the conversion from bfloat16 to float4, if the input is inf of the bfloat16 type or is out of range of the fp4x2_e2m1_t or fp4x2_e1m2_t type, the output will be the maximum or minimum value of the fp4x2_e2m1_t or fp4x2_e1m2_t type with the corresponding sign. If the input is nan, the output of the fp4x2_e2m1_t or fp4x2_e1m2_t type is 0.
    • For the conversion from bfloat16_t to fp4x2_e2m1_t or fp4x2_e1m2_t, the instruction reads and writes every two elements as a pair. The valid bits of mask are even-numbered bits. For example, 10 00 00 00 10 10 00 10 is equivalent to 11 11 00 00 10 10 00 11.

    • For the fp8_e8m0_t type:

      If the input is bfloat16_t +/–inf or the absolute value exceeds the maximum value of the fp8_e8m0_t type, the maximum value will be 0b11111110 of the fp8_e8m0_t type.

      If the input is nan of the bfloat16_t type, the output will be fp8_e8m0_t nan = 0b11111111.

  • Integer to integer

    Unsaturated mode: The input is truncated to the target data format. For example, if the input int32_t value is 256, the output uint8_t value will be 0.

    Saturation mode: If the input is out of the target data range, it will be saturated to the maximum or minimum value of the target data type.

    For the conversion from a narrow data type (such as int16_t, 2 bytes) to a wide data type (such as uint32_t, 4 bytes), only the saturation mode is supported. A negative input value will be saturated to 0.

Example

__simd_vf__ inline void CastVF(__ubuf__ int16_t* dstAddr, __ubuf__ float* srcAddr, uint32_t count, uint32_t srcRepeatSize, uint32_t dstRepeatSize, uint16_t repeatTimes)
{
    // Add static to castTrait variables.
    static constexpr AscendC::Reg::CastTrait castTrait = 
    {AscendC::Reg::RegLayout::ZERO, AscendC::Reg::SatMode::NO_SAT,AscendC::Reg::MaskMergeMode::ZEROING,AscendC::RoundMode::CAST_RINT};
   
   
    
    AscendC::Reg::RegTensor<float> srcReg;
    AscendC::Reg::RegTensor<int16_t> dstReg;
    AscendC::Reg::MaskReg mask;
    for (uint16_t i = 0; i < repeatTimes; i++) {
        AscendC::Reg::LoadAlign(srcReg, srcAddr + i * srcRepeatSize);
        mask = AscendC::Reg::UpdateMask<float>(count);
        AscendC::Reg::Cast<int16_t, float, castTrait>(dstReg, srcReg, mask);
        AscendC::Reg::StoreAlign(dstAddr + i * dstRepeatSize, dstReg, mask);
    }
}
Parent topic: Type Conversion
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details