CountBatch

Applicability

Product	Supported
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Function Description

CountBatch is used to combine multiple data records into batches by batch_size based on the UDF as compute ProcessPoints.

This function applies to the asynchronous dataflow scenario. The details are as follows:

If no data is input for a long time, you can use the CountBatch function to set the timeout period. If padding is not set, the existing data will be sent to compute ProcessPoints for processing after timeout.
After the timeout period is set, if the data is less than batch_size, you can set the padding attribute by using the CountBatch function. The compute ProcessPoint fills the data to batch_size based on the padding setting and then outputs the data.

Usage

When constructing a graph in DataFlow, you can set attributes for inputs of FlowNode to add the CountBatch function. The following is an example:

CountBatch count_batch = {};
// Set attributes in count_batch as needed.
count_batch.timeout = 10;
count_batch.batch_size = 300;
count_batch.padding = true;
count_batch.slide_stride = 5; 
DataFlowInputAttr flow_attr = {DataFlowAttrType::COUNT_BATCH, &count_batch};
std::vector<DataFlowInputAttr> flow_attrs = {flow_attr};
// Set MapInput of FlowNode.
FlowNode::MapInput(xx, xx, xx, flow_attrs);

Attribute	Data Type	Description
batch_size	int64_t	Size of a batch to be combined.
timeout	int64_t	This parameter takes effect only when batch_size is set. Wait time for combining batches, in ms. The value range is [0, 4294967295). The default value is 0, indicating that the group waits until the batch is full.
padding	bool	This parameter takes effect only when batch_size and timeout are set. Whether to perform padding when the batch is insufficient. The default value is false, indicating that padding is not performed.
slide_stride	int64_t	This parameter takes effect only when batch_size is set. Sliding window stride. The value range is [0, batchsize]. When the value is greater than 0 but less than batch_size, batches are combined based on the mode in which the sliding window is enabled. If this attribute is not set, the value is 0. If the value is equal to batch_size, batches are combined based on the mode in which the sliding window stride is not set. An error is reported if the value is greater than batch_size.
batch_dim	int64_t	Not enabled.
flag	int32_t	Not enabled.
drop_remainder	bool	Not enabled.

Precautions

Currently, the Batch feature cannot be used for load sharing. Therefore, if the 2P environment is used, you need to add {"ge.exec.logicalDeviceClusterDeployMode", "SINGLE"} and {"ge.exec.logicalDeviceId", "[0:0]"} during GE initialization.

Parent topic: Struct DataFlowInputAttr