dataflow.TimeBatch

Applicability

Product	Supported
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Function Description

The TimeBatch function is based on the UDF.

In normal scenarios, a model processes a piece of data each time. To process multiple pieces of data at a time, you need to combine the data into a batch. A basic way is to directly combine the data (N pieces), and add N before the shape. In some scenarios, data of one or more time segments needs to be combined into a batch based on specific dimensions. In this case, you can use TimeBatch.

In the Automatic Speech Recognition (ASR) scenario, there are two ways to combine data into a batch: by fixed-length time segment or by discontinuous time segment. You can use TimeBatch to implement them.

Prototype

TimeBatch(time_window=0, batch_dim=0, drop_remainder=False)

Parameters

Parameter	Data Type	Description
time_window	int64_t	The value is an integer (in ms). If the value is greater than 0, batches are combined based on the time window. If the value is -1, batches are combined based on the time segment. An error will be reported if the value is other values.
batch_dim	int64_t	This parameter takes effect only when time_window is set. The value range is [-1, shape dimension]. The default value is -1, indicating that a batch dimension is added to dimension 0 for the output shape. If the shape dimension is greater than batch_dim and batch_dim is greater than or equal to 0, a batch will be combined based on a certain dimension. An error will be reported if batch_dim is greater than the shape dimension or less than -1.
drop_remainder	Bool	This parameter takes effect only when time_window is set. It takes effect only when time_window is greater than 0. It determines whether to drop the remainder when time_window is insufficient. It is set to false by default, indicating that it is not to drop the remainder. The value true indicates that the remainder is dropped. Examples: Assume that time_window is 5 ms and the input data duration is 3 ms. If drop_remainder is not set or is set to false, the input data will not be dropped. If drop_remainder is set to true: If the input data does not carry EOS or SEG, the system will keep waiting and not drop the data. If the input data carries only the SEG, the data will be dropped. If the input data carries the EOS flag, the data will be dropped ,and only the EOS flag will be transferred.

Parameter

Data Type

Description

time_window

int64_t

The value is an integer (in ms). If the value is greater than 0, batches are combined based on the time window. If the value is -1, batches are combined based on the time segment. An error will be reported if the value is other values.

batch_dim

int64_t

This parameter takes effect only when time_window is set. The value range is [-1, shape dimension].

The default value is -1, indicating that a batch dimension is added to dimension 0 for the output shape.
If the shape dimension is greater than batch_dim and batch_dim is greater than or equal to 0, a batch will be combined based on a certain dimension.
An error will be reported if batch_dim is greater than the shape dimension or less than -1.

drop_remainder

Bool

This parameter takes effect only when time_window is set.

It takes effect only when time_window is greater than 0. It determines whether to drop the remainder when time_window is insufficient. It is set to false by default, indicating that it is not to drop the remainder. The value true indicates that the remainder is dropped. Examples:

Assume that time_window is 5 ms and the input data duration is 3 ms.

If drop_remainder is not set or is set to false, the input data will not be dropped.
If drop_remainder is set to true:
- If the input data does not carry EOS or SEG, the system will keep waiting and not drop the data.
- If the input data carries only the SEG, the data will be dropped.
- If the input data carries the EOS flag, the data will be dropped ,and only the EOS flag will be transferred.

Returns

None is returned in normal scenarios.

TypeError is returned when the parameter type is incorrect.

Examples

import dataflow as df
# Set each attribute value in time_batch as required.
time_batch = df.TimeBatch()
time_batch.time_window = 5
time_batch.batch_dim = 0
# Use the map_input API of FlowNode.
df.FlowNode(...).map_input(..., [time_batch])

Constraints

Currently, the Batch feature cannot be used for load sharing. Therefore, if the 2P environment is used, you need to add {"ge.exec.logicalDeviceClusterDeployMode", "SINGLE"}, {"ge.exec.logicalDeviceId", "[0:0]"} during ge initialization. The value of logicalDeviceId can be [0:0] or [0:1]. logicalDeviceId is described as follows:

Device on which the model is deployed at logical_device_cluster_deploy_mode = SINGLE.

Format: [node_id:device_id]

node_id: logical ID of the AI processor, starting from 0 and indicating the sequence number of the device in the resource configuration file.
device_id: physical ID of the AI processor.

Parent topic: Graph Construction APIs