--singleop

Applicability

Description

Converts a single-operator description JSON file into an offline model adapted to the AI processor for operator functional verification.

Compatibility:

In dynamic-shape operator scenarios and Ascend virtual instance scenarios, the CANN package version of the OM offline model conversion environment must be the same as that of the operating environment.

(The Ascend virtual instance is used in the NPU resource virtualization scenario, which aims to improve the utilization of physical NPU resources through isolated vNPU instances. NPU is short for neural network processing unit and vNPU is short for virtual NPU.)

See Also

This option works with the following options, of which, --output and --soc_version are required.

• Atlas 200I/500 A2 inference product :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --sparsity
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas inference product :
  • --output
  • --soc_version
  • --core_type
  • --aicore_num
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas training product :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas A2 training product / Atlas A2 inference product :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --sparsity
  • --allow_hf32
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas A3 training product / Atlas A3 inference product :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --sparsity
  • --allow_hf32
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• SoC:
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --sparsity
  • --allow_hf32
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip

Argument

Argument: single-operator description JSON file. For details about the format and configuration of the file, see Generating a Single-Operator Model.

Restrictions: The specified single operator is defined based on Ascend IR. For details about the single-operator definitions, see "Ascend IR Operator Specifications".

Suggestions and Benefits

None

Example

The following uses the single operator Add as an example for illustration. Upload the operator description file add.json to any directory on the ATC server, for example, $HOME/singleop. The following is an example:

--singleop=$HOME/singleop/add.json --output=$HOME/singleop/out/op_model  --soc_version=<soc_version>

Dependencies and Restrictions

• Option Usage:
  To convert a single-operator JSON file into an offline model and only use TBE operators during the conversion process (errors are reported if TBE operators cannot be found, without searching for AI CPU operators), set the following environment variable:

  export ASCEND_ENGINE_PATH=${INSTALL_DIR}/lib64/plugin/opskernel/libfe.so:${INSTALL_DIR}/lib64/plugin/opskernel/libge_local_engine.so:${INSTALL_DIR}/lib64/plugin/opskernel/librts_engine.so

  If you want to perform other operations after running the preceding command, run the unset ASCEND_ENGINE_PATH command to disable the environment variable.

  Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

• API Usage:

  When using an OM offline model file converted from a single-operator description file in an application project for model inference, call the single-operator model acceleration API (for example, aclopSetModelDir) to load the operator model, and then call the single-operator execution API (for example, aclopExecuteV2) to execute the operators.

  For details about the API, see "Single-Operator Model Execution".