aclmdlSetExternalWeightAddress

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product / Atlas A3 inference product	√
Atlas A2 training product / Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product	√
Atlas training product	√

Function Usage

Configures device memory for storing external weights. The caller needs to copy external weight data to the memory before model loading. If there are multiple external weight files, this API needs to be called multiple times.

During model loading, each FileConstant node in the computational graph stores the path of the corresponding external weight file (for example, /xx/../weight_hasid1). The graph engine (GE) checks whether device memory for external weights is configured based on the file name (for example, weight_hasid1) in the path. If device memory is configured, the memory address is used. Otherwise, GE allocates a device memory block, reads the external weight file, and copies the external weight data to the device memory. The memory is freed during model unloading.

This API needs to work with the following APIs to load models:

Call aclmdlCreateConfigHandle to create a configuration object for loading a model.
(Optional) Call aclmdlSetExternalWeightAddress to configure device memory for storing external weights.
Call aclmdlSetConfigOpt for multiple times to set the value of each attribute in the configuration object.
Call aclmdlLoadWithConfig to specify the configuration information for model loading and load your model.
After the model is successfully loaded, call aclmdlDestroyConfigHandle to destroy the configuration object.

Prototype

      
           aclError aclmdlSetExternalWeightAddress(aclmdlConfigHandle *handle, const char *weightFileName, void *devPtr, size_t size)

Parameters

Parameter	Input/Output	Description
handle	Output	Pointer to the configuration object for model loading. For details about the type definition, see aclmdlConfigHandle. Call aclmdlCreateConfigHandle to create an object in advance.
weightFileName	Input	Name of an external weight file, excluding the path. Generally, if the size of an OM model file is limited or the model file is encrypted, you need to load the weight file separately. Therefore, you need to save the weight in a separate file during model building. When you use ATC to generate an OM file, set --external_weight to 1, indicating that the weight of the Const/Constant node on the original network is saved in a separate file in the weight directory.
devPtr	Input	Pointer to the memory address for storing external weights on the device. For details about the memory allocation API, see Memory Management. The device memory is managed by the user. It cannot be freed during model execution.
size	Input	Memory size, in bytes. The value must be 32-byte aligned.

Returns

0 on success; else, failure. For details, see aclError.

Parent topic: Model Loading and Unloading