Converting a SavedModel to an .om Model

Overview

This section describes how to use the saved_model2om.py tool to reconstruct a trained model in SavedModel format and convert it into a SavedModel for loading the OM model based on the NPU version. Using the generated SavedModel during TF Serving deployment shortens the compile time and improve the TF Serving deployment performance.

When an .om model is used for online inference, the data dump function of Model Accuracy Analyzer is not supported.

Options

**Table 1** Options
Option	Description	Example Value
--input_path	Input path of the original SavedModel file. Required.	/home/HwHiAiUser/inputpath/model
--output_path	Output path of the SavedModel file generated after conversion. Required.	/home/HwHiAiUser/outputpath/model
--input_shape	Shape value of the input model. The format is name1:shape;name2:shape;name3:shape. When input_shape is set, the dimensions that are not clearly defined in the shape are automatically set to 1. Optional.	input:16,224,224,3
--soc_version	SoC version of the output .om model. You do not need to set it if --profiling is set. The value is determined by the device that performs the conversion. Required.	Ascendxxx
--profiling	When this option is set, AOE optimization is enabled. (You do not need to specify job_type.) 1: enables subgraph tuning. 2: enables operator tuning. Set this option if subgraph or operator tuning is required.	1
--method_name	Inference method when the TF Serving runtime is configured. If this parameter is not specified, the inference method is obtained from the original SavedModel file. Optional.	/tensorflow/serving/predict
--new_input_nodes	Re-selects an input node. The format is "Operator:Type:Operator name;Operator:Type:Operator name". Optional.	embedding:DT_FLOAT:bert/embedding/word_embeddings:0;add:DT_INT:bert/embedding/add:0
--new_output_nodes	Re-selects an output node. The format is "Operator:Operator name". Optional.	loss:loss/Softmax:0
--output_type	Sets the output data type of the network or an output node. For details about how to use the option, see AOE Instructions or ATC Instructions. Optional.	node1:0:FP16
--input_fp16_nodes	Sets the name of the input node whose input data type is FP16. For details about how to use the option, see AOE Instructions or ATC Instructions. Optional.	node_name1;node_name2

This tool supports both ATC and AOE options.

If --profiling is set, see AOE Instructions.
If --profiling is not set, see ATC Instructions.

This tool does not support the --out_nodes, --is_input_adjust_hw_layout, and --is_output_adjust_hw_layout options of the ATC and AOE tools. The --out_nodes option can be replaced by the --new_output_nodes option in Table 1.

Conversion

Download the conversion tool saved_model2om.py to any directory on the server, for example, $HOME/tools/. You do not need to install the tool.
Run the following command to perform the conversion. Modify the options based on the actual situation.
```
python3 saved_model2om.py --input_path "/home/HwHiAiUser/inputpath/model" --output_path "/home/HwHiAiUser/outputpath/model" --input_shape "input:16,224,224,3" --soc_version "Ascendxxx"
```
To query the value of --soc_version, perform the following steps:
- Run the npu-smi info command on the server where the Ascend AI Processor is installed to obtain the Chip Name information. The actual value is AscendChip Name. For example, if Chip Name is xxxyy, the actual value is Ascendxxxyy.
If subgraph or operator tuning is required during the conversion, run the following command:
```
python3 saved_model2om.py --input_path "/home/HwHiAiUser/inputpath/model" --output_path "/home/HwHiAiUser/outputpath/model" --input_shape "input:16,224,224,3" --profiling "1"
```
After the conversion is successful, a SavedModel file for loading the OM model is generated in the specified output_path. The file name format is {om_name}_load_om_saved_model_{timestamp}.

Parent topic: Sample Reference