Generating Operators

Procedure

1. Go to the installation directory mxIndex-{version}. For details about the directory and file names, see Table 1.

   cd mxIndex-{version}

   Table 1 Index SDK directory and file names

   Directory or File Name	Description
   device	Dynamic library and header file containing the IndexIL algorithm.
   filelist.txt	Software package files.
   host	Retrieval dynamic library, used for feature retrieval.
   include	API header file.
   lib	Retrieval dynamic library that links to host/lib.
   modelpath	Directory for storing the operator OM file. After an operator is built, store the OM file in this folder.
   ops	Contains the custom_opp_<arch>.run script, which is used to install retrieval algorithm-related operators.
   script	Contains the uninstallation script uninstall.sh, which is used to uninstall the Index SDK installation package.
   tools	Contains Python scripts used for operator generation.
   version.info	Version information.

2. Go to the ops directory. Before building an operator, set ASCEND_HOME, ASCEND_VERSION, and ASCEND_OPP_PATH to ~/Ascend, ~/ascend-toolkit/latest, and ~/Ascend/ascend-toolkit/latest/opp, respectively.

   1 2 3

   export ASCEND_HOME=~/Ascend export ASCEND_VERSION=~/Ascend/ascend-toolkit/latest export ASCEND_OPP_PATH=~/Ascend/ascend-toolkit/latest/opp

   • ASCEND_HOME indicates the path for storing files after CANN Toolkit is installed.
   • ASCEND_VERSION indicates the current Ascend version. If the ATC installation path is /usr/local/Ascend/ascend-toolkit/latest, you do not need to set ASCEND_HOME and ASCEND_VERSION.
   • ASCEND_OPP_PATH indicates the root directory of the operator library. You must have the write permission on the directory.

   

   MAX_COMPILE_CORE_NUMBER is used to specify the number of CPU cores available for graph compilation. It is used during operator running. The default value is 1. You do not need to set it.

3. Run the script based on the actual system architecture.
   • ARM

     1	./custom_opp_aarch64.run

   • x86_64

     1	./custom_opp_x86_64.run

   When running a script command, you can enter optional options. For details, see Table 2.

   Table 2 Options in custom_opp_{arch}.run

   Option	Description
   --help | -h	Queries help information.
   --info	Queries the package building information.
   --list	Queries the file list.
   --check	Checks package integrity.
   --quiet|-q	(Optional) Indicates silent installation. If it is used, Huawei Enterprise End User License Agreement (EULA) is agreed by default. The agreement signing process is skipped, and the printed man-machine interaction information is reduced.
   --nox11	Discarded
   --noexec	Decompresses the software package to the current directory without running the installation script. It is used together with --extract=<path> in the format of --noexec --extract=<path>.
   --extract=<path>	Decompresses the software package to a specified directory. It can be used with --noexec.
   --tar arg1 [arg2 ...]	Runs the tar command on the software package. Use the options following tar as the command options. For example, the --tar xvf command indicates that the .run package will be decompressed to the current directory.

   

   The following options are not displayed in --help. Do not use them directly.

   • --xwin: uses the xwin operating mode.
   • --phase2: performs the second step.
4. Go to the tools directory and generate the required operator. Before generating an operator, ensure that the CANN dependencies have been installed.
   • To generate only operators required by the used algorithm, determine the target operators by referring to Algorithm Introduction and then generate them by referring to Custom Operator.
   • To generate operators of all algorithms in batches by referring to Table 3.

     Table 3 Generating operators in batches

     Usage	python3 run_generate_model.py -m <mode> -t <npu_type> -p <pipeline> -pool <pool_size>
     Parameter	<mode>: algorithm mode. The value can be All, Flat, SQ8, IVFSQ8, or INT8. Use commas (,) to separate multiple values, for example, python3 run_generate_model.py -m Flat,IVFSQ8. By default, all modes are selected. You can directly executed python3 run_generate_model.py. <npu_type> indicates the processor name. For the Atlas 200/300/500 inference product and Atlas inference product, run the npu-smi info command on the server where the AI processor is installed and then delete the last digit of Name. The obtained value is the value of npu_type. For the Atlas 800I A2 inference server, run the npu-smi info command on the server where the AI processor is installed. The value of Name is the value of npu_type. For the Atlas 800I A3 SuperPoD Server, run the npu-smi info -t board -i 0 -c 0 command to obtain the NPU Name information. 910_NPU Name is the value of npu_type. <pipeline>: whether to use multi-thread parallel pipelines to generate an operator model. The default value is true. If this option is set to true, the default value 32 of pool_size is used. <pool_size>: size of the process pool for multi-process scheduling of operators generated in batches. --help | -h: help information.
     Description	Run the command to obtain multiple groups of operator model files. Before running the command, you need to modify the para_table.xml file in the current directory and fill the required parameters in the table. 1 ≤ pool_size ≤ 32

     

     The constraints in the operator generation description table indicate the option combinations that are frequently used in services. For details about the running exceptions occurred when other options are used, see CANN ATC Offline Model Compilation Tool User Guide.

5. Prepare the operator model file.
   • You can set the model file directory to the environment variable MX_INDEX_MODELPATH. The environment variable can be a path starting with ~, a relative path, or an absolute path. However, the path cannot contain soft links. When this variable is used, it is converted to an absolute path and restricted to the /home or /root path.

     1 2	mv op_models/* $PWD/../modelpath export MX_INDEX_MODELPATH=`realpath $PWD/../modelpath`

   • If the environment variable is not used for configuration, move the operator model file to the modelpath directory.

   After the operator is generated, keep the OM file secure and ensure that the file is not tampered with.

   

   If the error message "Failed to import Python module" is displayed during operator generation, rectify the fault by referring to np.float_ Has Been Removed from NumPy Data Types.