Converting a SavedModel to an .om Model

Description

This section describes how to use the saved_model2om.py tool to convert a trained SavedModel into an .om model. Using the converted .om model during TF Serving deployment shortens the compilation time and improves the TF Serving deployment performance.

When an .om model is used for online inference, the data dump function for accuracy comparison is not supported.

Options

**Table 1** Options
Option	Description	Example Value
--input_path	Input path of the original SavedModel file. Required.	$HOME/inputpath/model
--output_path	Output path of the SavedModel file generated after conversion. Required.	$HOME/outputpath/model
--input_shape	Shape value of the input model. The format is name1:shape;name2:shape;name3:shape. When input_shape is set, the dimensions that are not clearly defined in the shape are automatically set to 1. Optional.	input:16,224,224,3
--soc_version	SoC version of the output .om model. You do not need to set it if --profiling is set. The value is determined by the device that performs the conversion. Required.	Ascendxxx
--profiling	When this option is set, AOE optimization is enabled. (You do not need to specify job_type.) 1: enables subgraph tuning. 2: enables operator tuning. Set this option if subgraph or operator tuning is required.	1
--method_name	Inference method when the TF Serving runtime is configured. If this parameter is not specified, the inference method is obtained from the original SavedModel file. Optional.	/tensorflow/serving/predict
--new_input_nodes	Re-selects an input node. The format is "Operator:Type:Operator name;Operator:Type:Operator name". Optional.	embedding:DT_FLOAT:bert/embedding/word_embeddings:0;add:DT_INT:bert/embedding/add:0
--new_output_nodes	Re-selects an output node. The format is "Operator:Operator name". Optional.	loss:loss/Softmax:0
--output_type	Sets the output data type of the network or an output node. For details about how to use the option, see AOE Tuning Tool or ATC. Optional.	node1:0:FP16
--input_fp16_nodes	Sets the name of the input node whose input data type is FP16. For details about how to use the option, see AOE Tuning Tool or ATC. Optional.	node_name1;node_name2

This tool supports both ATC and AOE parameters.

If --profiling is set, see AOE Tuning Tool.
If --profiling is not set, see ATC.

This tool does not support the --out_nodes, --is_input_adjust_hw_layout, and --is_output_adjust_hw_layout options of the ATC and AOE tools. The --out_nodes option can be replaced by the --new_output_nodes option in Table 1.

Conversion

Download the conversion tool saved_model2om.py to any directory on the server, for example, $HOME/tools/. You do not need to install the tool.
Run the following command to perform the conversion. Modify the parameters based on the actual situation.
```
python3 saved_model2om.py --input_path "$HOME/inputpath/model" --output_path "$HOME/outputpath/model" --input_shape "input:16,224,224,3" --soc_version "Ascendxxx"
```
To query the value of --soc_version, perform the following steps:
- For the following Product: Run the npu-smi info command on the server where AI processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.
  Atlas A2 training product/Atlas A2 inference product
  
  Atlas inference product
  
  Atlas training product
- For the following Product: Run the npu-smi info -t board -i id -c chip_id command on the server where AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234. Where:
  - id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
  - chip_id: chip ID, which is obtained by running the npu-smi info -m command.
  Atlas 350 Accelerator Card
  
  Atlas A3 training product/Atlas A3 inference product
If subgraph or operator tuning is required during the conversion, run the following command:
```
python3 saved_model2om.py --input_path "$HOME/inputpath/model" --output_path "$HOME/outputpath/model" --input_shape "input:16,224,224,3" --profiling "1"
```
After the conversion is successful, a SavedModel file for loading the OM model is generated in the specified output_path. The file name format is {om_name}_load_om_saved_model_{timestamp}.

Parent topic: Common Operations