TimeBatch

Applicability

Product	Supported
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Function Description

The TimeBatch function is based on the UDF.

A model normally processes a piece of data each time. When it needs to process many pieces of data at a time, these pieces of data need to be combined into a batch. The basic mode is to combine the N pieces of data and add N before the shape. In some scenarios, data of one or more time segments needs to be combined into a batch based on a specific dimension, which can be implemented using the TimeBatch function.

In the Automatic Speech Recognition (ASR) scenario, there are two ways to combine data into a batch: by fixed-length time segment or by discontinuous time segment. You can use TimeBatch to implement them.

Usage

When constructing a graph in DataFlow, you can set attributes for inputs of FlowNode to add the TimeBatch function. The following is an example:

TimeBatch time_batch = {};
// Set attributes in time_batch as needed.
time_batch.time_window = 10;
time_batch.batch_dim = 5;
time_batch.drop_remainder = true;
DataFlowInputAttr flow_attr = {DataFlowAttrType::TIME_BATCH, &time_batch};
std::vector<DataFlowInputAttr> flow_attrs = {flow_attr};
// Set MapInput of FlowNode.
FlowNode::MapInput(xx, xx, xx, flow_attrs);

Attribute	Data Type	Description
time_window	int64_t	The value is an integer (in ms). If the value is greater than 0, batches are combined based on the time window. If the value is -1, batches are combined based on the time segment. An error will be reported if the value is other values.
batch_dim	int64_t	This attribute takes effect only when time_window is set. The value range is [-1, shape dimension]. The default value is -1, indicating that a batch dimension is added to dimension 0 for the output shape. If the shape dimension is greater than batch_dim and batch_dim is greater than or equal to 0, a batch will be combined based on a certain dimension. An error will be reported if batch_dim is greater than the shape dimension or less than -1.
drop_remainder	bool	This attribute takes effect only when time_window is set. It takes effect only when window is greater than 0. It determines whether to drop the remainder when window is insufficient. It is set to false by default, indicating that it is not to drop the remainder. The value true indicates that the remainder is dropped. An example is as follows: Assume that time_window is 5 ms and the input data duration is 3 ms. If drop_remainder is not set or is set to false, the input data will not be dropped. If drop_remainder is set to true: If the input data does not carry EOS or SEG, the system will keep waiting and not drop the data. If the input data carries only the SEG, the data will be dropped. If the input data carries the EOS flag, the data will be dropped ,and only the EOS flag will be transferred.
time_interval	int64_t	Not enabled.
timeout	int64_t	Not enabled.
flag	int32_t	Not enabled.
padding	bool	Not enabled.

Precautions

Currently, the Batch feature cannot be used for load sharing. Therefore, if the 2P environment is used, you need to add {"ge.exec.logicalDeviceClusterDeployMode", "SINGLE"} and {"ge.exec.logicalDeviceId", "[0:0]"} during GE initialization. The value of logicalDeviceId can be [0:0] or [0:1]. The details are as follows:

Device on which the model is deployed when logicalDeviceClusterDeployMode is set to SINGLE.

Format: [node_id:device_id]

node_id: logical ID of the AI processor, starting from 0 and indicating the sequence number of the device in the resource configuration file.
device_id: physical ID of the AI processor.

Parent topic: Struct DataFlowInputAttr