General Restrictions and APIs

General Restrictions

AMCT does not perform security verification of passed file paths. Ensure that the file path arguments passed to any API call are correct.
If the API contains a parameter that requires the user to enter the file path, the value of the parameter will be overwritten when quantization is performed again. The quantization log printed to the screen also contains a warning message indicating that the related file is overwritten.

APIs

Category	API	Function
Common API	ModelEvaluator	Provides a Python instance based on the input data of the bin type for a model to calibrate and infer the model.
Torch module-based quantization APIs	quantize	Converts the high-precision model into a quantization calibration model and calculates the quantization parameters after inference.
	convert	Converts a quantization calibration model into a quantized deployable model.
	algorithm_register	Registers the custom algorithm provided by the user with AMCT.
PTQ APIs	create_quant_config	Finds all quantizable layers in a graph, creates a quantization configuration file, and writes the quantization configuration of the quantizable layers to the configuration file.
	quantize_model	Quantizes a graph based on the quantization configuration file, inserts the weight and activation quantization operators, generates a quantization factor record file record_file, and returns a torch.nn.Module model ready for calibration.
	save_model	Inserts operators such as AscendQuant and AscendDequant into the modified model based on the quantization factor record file record_file and generates a fake-quantized model for accuracy simulation in the ONNX Runtime environment and a model deployable on the AI processor for inference.
	accuracy_based_auto_calibration	Calibrates the input model based on the input configuration file to search for a quantization configuration that meets accuracy requirements, and outputs a fake-quantized model for accuracy simulation in the ONNX Runtime environment and a model deployable on the AI processor for inference.
	quantize_preprocess	Preprocesses the quantization of a graph based on the quantization configuration file, inserts the balanced quantization operators, generates a balanced quantization factor record file record_file, and returns a torch.nn.Module model ready for calibration.
QAT APIs	create_quant_retrain_config	Finds all quantizable layers in a graph, creates a quantization configuration file, and writes the quantization configuration of the quantizable layers to the configuration file.
	create_quant_retrain_model	Quantizes a graph based on the given configuration file, inserts quantization-related layers (quantization-aware layers of activations and weights and layers for searching for N), generates a quantization factor record file (record_file), and returns the resultant model of the torch.nn.Module type for QAT.
	restore_quant_retrain_model	Quantizes a graph based on the given configuration file, inserts quantization-related layers (quantization-aware layers of activations and weights and layers for searching for N), generates a quantization factor record file (record_file), loads the checkpoint weight parameters saved during training, and returns the resultant model of the torch.nn.Module type.
	save_quant_retrain_model	Inserts operators such as AscendQuant and AscendDequant into the retrained model and generates a fake-quantized model for accuracy simulation and a deployable model.
QAT APIs (single-operator mode)	LinearQAT	Constructs the QAT operator of Linear.
	Conv2dQAT	Constructs the QAT operator of Conv2d.
	ConvTranspose2dQAT	Constructs the QAT operator of ConvTranspose2d.
	Conv3dQAT	Constructs the QAT operator of Conv3d.
Sparsity APIs	create_prune_retrain_model	Filter-level sparsity or 2:4 structured sparsity API. Only either of the two sparsity features can be enabled at a time. This API sparsifies the input graph based on the given sparsity configuration file, inserts or replaces related operators in the input graph, generates a sparsity record file record_file, and returns a resultant torch.nn.Module model that can be used for retraining.
	restore_prune_retrain_model	Filter-level sparsity or 2:4 structured sparsity API. Only either of the two sparsity features can be enabled at a time. This API sparsifies the input graph based on the given sparsity record file record_file, and returns a resultant torch.nn.Module model that can be used for retraining.
	save_prune_retrain_model	Generates the final ONNX fake-quantized model and deployable model based on the retrained sparse model.
Auto channel pruning search API	auto_channel_prune_search	Calculates the sparsity sensitivity (affecting accuracy) and sparsity gain (affecting performance) of each channel based on the user model. Then, the search policy searches for the optimal layer-wise channel sparsity ratio based on the input to balance accuracy and performance. Finally, a configuration file is generated.
Compression combination APIs	create_compressed_retrain_model	Applies to static compression combination. Compresses the input model based on the specified static compression combination configuration file. That is, prunes the input model (via either filter-level sparsity or 2:4 structured sparsity), inserts quantization operators (QAT layer for activations and weights and searchN layer) into the model, generates the sparsity record file record_file (if the configuration exists), and returns the modified torch.nn.Module model.
	restore_compressed_retrain_model	Compresses the input model to be statically combined based on the specified compression combination configuration file and record file (sparsification before quantization), and loads the saved weight. Sparsifies the input model based on the sparsity records in the given record_file, and then inserts quantization-related operators (QAT layer for activations and weights and searchN layer) into the model. Loads the checkpoint weight parameters saved during training and returns the modified torch.nn.Module model.
	save_compressed_retrain_model	Applies to static compression combination. Based on the retrained model, generates a fake-quantized model for accuracy simulation and a deployable model, which have undergone static compression combination.
Tensor decomposition APIs	auto_decomposition	(Optional) Performs tensor decomposition on the input PyTorch model object to obtain the model object after decomposition and the names of the layers before and after decomposition, and saves the decomposition information file.
Tensor decomposition APIs	decompose_network	Decomposes the input PyTorch model object based on the decomposition information file saved via the auto_decomposition API call and returns the decomposed model object and the layer names before and after decomposition.
Distillation APIs	create_distill_config	Finds all distillable layers and structures based on the graph structure, automatically generates a distillation configuration file, and writes the quantization configuration and distillation structure of the layers into a configuration file.
	create_distill_model	Quantizes the input graph structure based on the specified distillation configuration file, inserts quantization-related operators (distillation layer and searching N layer of activations and weights) into the input graph structure, and returns the modified torch.nn.Module model that can be used for distillation.
	distill	Distills the input graph structure based on the given distillation configuration file and returns the modified torch.nn.Module model.
	save_distill_model	Generates a fake-quantized model for accuracy simulation and a deployable model based on the distilled model.
KV cache quantization APIs	create_quant_cali_config	Generates the detailed quantization configuration (mainly about the KV cache) of each layer based on the model, quantization layer information, and quantization configuration input by users.
	create_quant_cali_model	Modifies the graph of the user's network model based on the model and detailed quantization configuration, replaces the Linear operator to be quantized with the output IFMR/HFMG quantization operator, calibrates the model online, and generates the quantization factors to be saved in record_file.
	QuantCalibrationOp	Calibrates the output by calling the IFMR/HFMG quantization algorithm based on the user's quantization algorithm configuration during forward propagation, and outputs the quantization factors to the specified layer name in the corresponding record file record_file. This API is used for graph construction.

Parent topic: API Description