aclmdlGetDynamicBatch

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	√
Atlas inference products	√
Atlas training products	√

Description

Obtains the dynamic batch size information supported by a model based on the model description.

Prototype

aclError aclmdlGetDynamicBatch(const aclmdlDesc *modelDesc, aclmdlBatch *batch)

Parameters

Parameter	Input/Output	Description
modelDesc	Input	Pointer to data of the aclmdlDesc type. Call aclmdlCreateDesc to create data of the aclmdlDesc type in advance.
batch	Output	const int ACL_MAX_BATCH_NUM = 128; typedef struct aclmdlBatch { size_t batchCount; /*Number of batch size profiles supported by the model/ uint64_t batch[ACL_MAX_BATCH_NUM]; /*Batch size profiles/ } aclmdlBatch; If batchCount is 0, dynamic batch size is not supported. The batch size defined in the model applies.

Parameter

Input/Output

Description

modelDesc

Input

Pointer to data of the aclmdlDesc type.

Call aclmdlCreateDesc to create data of the aclmdlDesc type in advance.

batch

Output

const int ACL_MAX_BATCH_NUM = 128;
typedef struct aclmdlBatch {
    size_t batchCount; /**Number of batch size profiles supported by the model*/
    uint64_t batch[ACL_MAX_BATCH_NUM]; /**Batch size profiles*/
} aclmdlBatch;

If batchCount is 0, dynamic batch size is not supported. The batch size defined in the model applies.

Returns

0 on success; else, failure. For details, see aclError.

Parent topic: aclmdlDesc