IVFSP

python3 ivfsp_generate_model.py --cores <core_num> -d <dim> -nonzero_num <low_dim> -nlist <k> -handle_batch <handle_batch> -code_num <code_num> -p <process_id> --pool <pool_size>

<core_num>: number of AI Cores. The default value is 8. You do not need to set this parameter.

<dim>: feature vector dimension. The default value is 256.

<low_dim>: number of non-zero dimensions after feature vector compression. The default value is 32.

<k>: number of cluster centroids. The value needs to be consistent with that of <k> in Generating the IVFSP Training Operator Model File, and defaults to 1024.

<handle_batch>: number of candidate buckets for each computing delivery during retrieval. The default value is 32.

<code_num>: maximum number of samples in each bucket for each computing during retrieval. If the bucket is too large, the program automatically splits the bucket into multiple operators based on code_num for distance calculation. The value needs to be consistent with that of <codebook_batch_size> in Generating the IVFSP Training Operator Model File, and defaults to 32768.

<process_id>: ID of the process for multi-process scheduling of operators generated in batches. The default value is 0, and you do not need to set this parameter.

<pool_size>: size of the process pool for multi-process scheduling of operators generated in batches. The default value is 16.

--help | -h: help information.

After running this command, you can obtain a group of AI Core operator model files used for IVFSP retrieval. You need to modify the parameters in the command.

• When dim ∈ {64, 128, 256}, k∈ {256, 512, 1024, 2048, 4096, 8192, 16384}; when dim ∈ {512, 768}, k∈ {256, 512, 1024, 2048}.
• The value of low_dim must be a multiple of 16 and less than or equal to min (128, dim).
• The value of handle_batch must be a multiple of 16 and meet the requirement:16 ≤ handle_batch ≤ 240.
• 0 < pool_size ≤ 32

python3 ivfsp_aicpu_generate_model.py --cores <core_num> -p <process_id>

<core_num>: number of AI Cores. The default value is 8. You do not need to set this parameter.

<process_id>: ID of the process for multi-process scheduling of operators generated in batches. The default value is 0, and you do not need to set this parameter.

--help | -h: help information.

After running this command, you can obtain a group of AICPU operator model files used for IVFSP retrieval.

python3 ivfsp_generate_pyacl_model.py --cores <core_num> -d <dim> -nonzero_num <low_dim> -nlist <k> -batch_size <batch_size> -code_num <codebook_batch_size> -p <process_id>

<core_num>: number of AI Cores. The default value is 8. You do not need to set this parameter.

<dim>: feature vector dimension. The default value is 256.

<low_dim>: number of non-zero dimensions after feature vector compression. The default value is 32.

<k>: number of cluster centroids. The value needs to be consistent with that of <k> in Generating the IVFSP Service Operator Model File, and defaults to 1024.

<batch_size>: Training is performed based on its value. The default value is 32768.

<codebook_batch_size>: maximum number of codebooks to be operated during training. The value must be a power of 2. The value needs to be consistent with that of <code_num> in Generating the IVFSP Service Operator Model File, and defaults to 32768.

<process_id>: ID of the process for multi-process scheduling of operators generated in batches. The default value is 0, and you do not need to set this parameter.

--help | -h: help information.

After running this command, you can obtain a group of operator model files used for IVFSP retrieval. You need to modify the parameters in the command. The generated IVFSP training operator model file is saved in the op_models_pyacl subdirectory of the current directory.

• When dim ∈ {64, 128, 256}, k∈ {256, 512, 1024, 2048, 4096, 8192, 16384}; when dim ∈ {512, 768}, k∈ {256, 512, 1024, 2048}.
• The value of low_dim must be a multiple of 16 and less than or equal to min (128, dim).
• The value of batch_size must be a multiple of 16.
• The value of codebook_batch_size must be a multiple of 16.