--singleop

Applicability

Description

Converts a single-operator description JSON file into an offline model adapted to the Ascend AI Processor for operator functional verification.

Compatibility:

In dynamic-shape operator scenarios and Ascend virtual instance scenarios, the CANN package version of the OM offline model conversion environment must be the same as that of the operating environment.

(The Ascend virtual instance is used in the NPU resource virtualization scenario, which aims to improve the utilization of physical NPU resources through isolated vNPU instances. NPU is short for neural network processing unit and vNPU is short for virtual NPU.)

See Also

This option works with the following options, of which, --output and --soc_version are required.

• Atlas 200I/500 A2 inference products :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --sparsity
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas inference products :
  • --output
  • --soc_version
  • --core_type
  • --aicore_num
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas training products :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas A2 training products / Atlas A2 inference products :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --sparsity
  • --allow_hf32
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip
• Atlas A3 training products / Atlas A3 inference products :
  • --output
  • --soc_version
  • --precision_mode
  • --precision_mode_v2
  • --op_select_implmode
  • --optypelist_for_implmode
  • --enable_small_channel
  • --sparsity
  • --allow_hf32
  • --log
  • --debug_dir
  • --op_compiler_cache_mode
  • --op_compiler_cache_dir
  • --atomic_clean_policy
  • --customize_dtypes
  • --is_weight_clip

Argument

Argument: single-operator description JSON file. For details about the format and configuration of the file, see Generating a Single-Operator Model.

Restrictions: The specified single operator is defined based on Ascend IR. For details about the single-operator definitions, see ""Ascend IR Operator Specifications"".

Suggestions and Benefits

None

Example

The following uses the single operator Add as an example for illustration. Upload the operator description file add.json to any directory on the ATC server, for example, $HOME/singleop. The following is an example:

--singleop=$HOME/singleop/add.json --output=$HOME/singleop/out/op_model  --soc_version=<soc_version>

Dependencies and Restrictions

• Restrictions:
  To convert a single-operator JSON file into an offline model and only use TBE operators during the conversion process (errors are reported if TBE operators cannot be found, without searching for AI CPU operators), set the following environment variable:

  export ASCEND_ENGINE_PATH=${INSTALL_DIR}/lib64/plugin/opskernel/libfe.so:${INSTALL_DIR}/lib64/plugin/opskernel/libge_local_engine.so:${INSTALL_DIR}/lib64/plugin/opskernel/librts_engine.so

  If you want to perform other operations after running the preceding command, run the unset ASCEND_ENGINE_PATH command to disable the environment variable.

  Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

• API Usage:

  When using an OM offline model file converted from a single-operator description file in an application project for model inference, call the acl API (for example, aclopSetModelDir) to load the operator model, and then call the acl API (for example, aclopExecuteV2) to execute the operators.

  For details about the APIs, see ""Single-Operator Model Execution"".