CountBatch

Applicable Products

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference products	x
Atlas training products	x

Function Description

CountBatch is used to combine multiple data records into batches by batch_size based on the UDF as compute ProcessPoints.

This function applies to the asynchronous dataflow scenario. The details are as follows:

If no data is input for a long time, you can use the CountBatch function to set the timeout period. If padding is not set, the existing data will be sent to compute ProcessPoints for processing after timeout.
After the timeout period is set, if the data is less than batch_size, you can set the padding attribute by using the CountBatch function. The compute ProcessPoint fills the data to batch_size based on the padding setting and then outputs the data.

Usage

When constructing a graph in DataFlow, you can set attributes for inputs of FlowNode to add the CountBatch function. The following is an example:

CountBatch count_batch = {};
// Set attributes in count_batch as needed.
count_batch.timeout = 10;
count_batch.batch_size = 300;
count_batch.padding = true;
count_batch.slide_stride = 5; 
DataFlowInputAttr flow_attr = {DataFlowAttrType::COUNT_BATCH, &count_batch};
std::vector<DataFlowInputAttr> flow_attrs = {flow_attr};
// Set MapInput of FlowNode.
FlowNode::MapInput(xx, xx, xx, flow_attrs);

Attribute	Data Type	Value Description
batch_size	int64_t	Size of a batch to be combined.
timeout	int64_t	This parameter takes effect only when batch_size is set. Wait time for combining batches, in ms. The value range is [0, 4294967295). The default value is 0, indicating that the group waits until the batch is full.
padding	bool	This parameter takes effect only when batch_size and timeout are set. Whether to perform padding when the batch is insufficient. The default value is false, indicating that padding is not performed.
slide_stride	int64_t	This parameter takes effect only when batch_size is set. Sliding window stride. The value range is [0, batchsize]. When the value is greater than 0 but less than batch_size, batches are combined based on the mode in which the sliding window is enabled. If this attribute is not set, the value is 0. If the value is equal to batch_size, batches are combined based on the mode in which the sliding window stride is not set. An error is reported if the value is greater than batch_size.
batch_dim	int64_t	Reserved.
flag	int32_t	Reserved.
drop_remainder	bool	Reserved.

Precautions

Currently, the Batch feature cannot be used for load sharing. Therefore, if the 2P environment is used, you need to add {"ge.exec.logicalDeviceClusterDeployMode", "SINGLE"} and {"ge.exec.logicalDeviceId", "[0:0]"} during GE initialization.

Parent topic: Struct DataFlowInputAttr