pypto.prelu

Applicable Products

• Ascend 950PR/Ascend 950DT: Supported
• Atlas A3 training products/Atlas A3 inference products: Supported
• Atlas A2 training products/Atlas A2 inference products: Supported

Description

Performs the parametric rectified linear unit (PReLU) operation on each element of input. When an element value is greater than or equal to 0, it remains unchanged; when it is less than 0, it is multiplied by a weight coefficient. The calculation formula is as follows:

$$res_i = \begin{cases} input_i & \text{if } input_i \geq 0 \\ weight_i \times input_i & \text{if } input_i < 0 \end{cases}$$

Here, weight is a one-dimensional tensor:

• When input is 1-dimensional, the length of weight is 1, and the weight is shared across elements.
• When input is 2- to 4-dimensional, the length of weight is the same as the size of the second dimension (channel dimension) of input, and the weight is shared across channels.

Prototype

prelu(input: Tensor, weight: Tensor) -> Tensor

Parameters

Return Value

Returns the output tensor, whose data type and shape are the same as those of input.

Constraints

1. The types of input and weight must be the same.
2. The shape of weight must be one-dimensional. When input is 1-dimensional, the length is 1; when input is 2- to 4-dimensional, the length is equal to the size of the second dimension of input.
3. Because temporary memory is used, when the input is two-dimensional, the TileShape size has an additional constraint. Assuming that TileShape is [a,b], then absizeof(self) + b/8 + 8KB < UB.
4. Tensor inputs do not support the TileOpFormat.TILEOP_NZ format.

Examples

TileShape Setting Example

Note: Before calling this operation API, set the TileShape through set_vec_tile_shapes.

The TileShape dimensions must be consistent with the output.

Example 1 (1D input): The input input and weight have shapes [n] and [1], respectively. The output is [n], and TileShape is set to [n1], where n1 is used to split the n axis.

Example 2 (2D input): The input input and weight have shapes [m, n] [n]. The output is [m, n], and TileShape is set to [m1, n1], where m1 and n1 are used to split the m and n axes, respectively.

pypto.set_vec_tile_shapes(4, 16)

API Call Example

Example 1: 1D input

# Example 1: 1D PReLU operation
# The input shape is [4], and the weight shape is [1].
# For negative elements, all elements share the same weight.
input_tensor = pypto.tensor([-2.0, 1.0, -3.0, 0.5], pypto.DT_FP32)
weight_tensor = pypto.tensor([0.25], pypto.DT_FP32)
out = pypto.prelu(input_tensor, weight_tensor)

The results are as follows:

Input data input:  [-2.0,  1.0, -3.0,  0.5]
Input data weight: [ 0.25 ]
Output data out:    [-0.5,  1.0, -0.75,  0.5]

Calculation process:

• All elements share the weight 0.25.
• -2.0 < 0, result = 0.25 × (-2.0) = -0.5; 1.0 ≥ 0, result = 1.0.
• -3.0 < 0, result = 0.25 × (-3.0) = -0.75; 0.5 ≥ 0, result = 0.5

Example 2: 2D Input

# Example 2: 2D PReLU operation.
# The input shape is [2, 3], and the weight shape is [3].
# For negative elements, multiply by the corresponding weight per channel.
input_tensor = pypto.tensor([[-2.0, 1.0, -3.0], [0.5, -1.0, 2.0]], pypto.DT_FP32)
weight_tensor = pypto.tensor([0.25, 0.5, 0.1], pypto.DT_FP32)
out = pypto.prelu(input_tensor, weight_tensor)

The results are as follows:

Input data input:  [[-2.0,  1.0, -3.0], [ 0.5, -1.0,  2.0]]
Input data weight: [ 0.25, 0.5,  0.1]
Output data out:    [[-0.5,  1.0, -0.3], [ 0.5, -0.5,  2.0]]

Calculation process:

• Channel 0: -2.0 < 0, result = 0.25 × (-2.0) = -0.5; 0.5 ≥ 0, result = 0.5
• Channel 1: 1.0 ≥ 0, result = 1.0; -1.0 < 0, result = 0.5 × (-1.0) = -0.5
• Channel 2: -3.0 < 0, result = 0.1 × (-3.0) = -0.3; 2.0 ≥ 0, result = 2.0