create_distill_config

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	√
Atlas inference product	√
Atlas training product	√

Description

Finds all distillable layers and structures based on the graph structure, automatically generates a distillation configuration file, and writes the quantization configuration and distillation structure of the layers into a configuration file.

Prototype

create_distill_config(config_file, model, input_data, config_defination=None)

Parameters

Parameter	Input/Output	Description
config_file	Input	Path (including the file name) of the distillation configuration file. The existing file (if any) in the path will be overwritten upon this API call. A string.
model	Input	Floating-point original model to be distilled, with weights loaded. A torch.nn.Module.
input_data	Input	Input data of the model. A torch.tensor is replaced with an equivalent tuple(torch.tensor). A tuple.
config_defination	Input	Simplified configuration file. The simplified configuration file distill.cfg is generated based on the distill_config_pytorch.proto file. The .proto file is stored in /amct_pytorch/proto/* under the AMCT installation directory. For details about the parameters in the .proto file and the generated simplified quantization configuration file distill.cfg, see Simplified Distillation Configuration File. Default: None* A string.

Returns

None

Example

import amct_pytorch as amct
# Build a graph of the network for distillation.
model = build_model()
model.load_state_dict(torch.load(state_dict_path))
input_data = tuple([torch.randn(input_shape)])
 
# Create a distillation configuration file.
amct.create_distill_config(config_file="./configs/config.json",
                           model,
                           input_data,
                           config_defination="./configs/distill.cfg")

Flush file:

A distillation configuration file in JSON format. (When distillation is performed again, the configuration file output by the API will be overwritten.) The following is an example configuration file of INT8 quantization:

{
    "version":1,
    "batch_num":1,
    "group_size":1,
    "data_dump":false,
    "distill_group":[
        [
            "conv1",
            "bn",
            "relu"
        ],
        [
            "conv2",
            "bn2",
            "relu2"
        ]
    ],
    "conv1":{
        "quant_enable":true,
        "distill_data_config":{
            "algo":"ulq_quantize",
            "dst_type":"INT8"
        },
        "distill_weight_config":{
            "algo":"arq_distill",
            "channel_wise":true,
            "dst_type":"INT8"
        }
    },
    "conv2":{
        "quant_enable":true,
        "distill_data_config":{
            "algo":"ulq_quantize",
            "dst_type":"INT8"
        },
        "distill_weight_config":{
            "algo":"arq_distill",
            "channel_wise":true,
            "dst_type":"INT8"
        }
    }
}

Parent topic: Distillation APIs