aclmdlSetExternalWeightAddress
Applicability
|
Product |
Supported |
|---|---|
|
|
√ |
|
|
√ |
|
|
√ |
|
|
√ |
|
|
√ |
Description
Configures device memory for storing external weights. The caller needs to copy external weight data to the memory before model loading. If there are multiple external weight files, this API needs to be called multiple times.
During model loading, each FileConstant node in the computational graph stores the path of the corresponding external weight file (for example, /xx/../weight_hasid1). The graph engine (GE) checks whether device memory for external weights is configured based on the file name (for example, weight_hasid1) in the path. If device memory is configured, the memory address is used. Otherwise, GE allocates a device memory block, reads the external weight file, and copies the external weight data to the device memory. The memory is freed during model unloading.
- Call aclmdlCreateConfigHandle to create a configuration object for loading a model.
- (Optional) Call aclmdlSetExternalWeightAddress to configure device memory for storing external weights.
- Call aclmdlSetConfigOpt multiple times to set the value of each attribute in the configuration object.
- Call aclmdlLoadWithConfig to specify the configuration information for model loading and load your model.
- After the model is successfully loaded, call aclmdlDestroyConfigHandle to destroy the configuration object.
Prototype
aclError aclmdlSetExternalWeightAddress(aclmdlConfigHandle *handle, const char *weightFileName, void *devPtr, size_t size)
Parameters
|
Parameter |
Input/Output |
Description |
|---|---|---|
|
handle |
Output |
Pointer to the configuration object for model loading. Call aclmdlCreateConfigHandle to create an object in advance. |
|
weightFileName |
Input |
Name of an external weight file, excluding the path. Generally, if the size of an .om model file is limited or the model file is encrypted, you need to load the weight file separately. Therefore, you need to save the weight in a separate file during model building. When you use ATC to generate an .om file, set --external_weight to 1, indicating that the weight of the Const/Constant node on the original network is saved in a separate file in the weight directory. |
|
devPtr |
Input |
Pointer to the memory address for storing external weights on the device. For details about the memory allocation API, see Memory Management. The device memory is managed by the user. It cannot be freed during model execution. |
|
size |
Input |
Memory size, in bytes. The value must be 32-byte aligned. |
Returns
0 on success; else, failure. For details, see aclError.