AscendIndexCodeBookInitFromMemParams

AscendIndexCodeBookInitFromMemParams (int numIter, int device, float ratio, int batchSize, int codeNum,bool verbose,std::string codeBookOutputDir,const float *memLearnData, size_t memLearnDataSize, bool isTrainAndAdd);

Initialization structure of the IVFSP training codebook. It loads training data from memory.

N/A

N/A

int numIter: number of training iterations. The default value is 1.

int device: logic device ID. The default value is 0.

float ratio: sampling rate of original samples used for training. The default value is 1.0.

int batchSize: batch size used for training. The value of this parameter must be the same as that of <batch_size> for generating the IVFSP training operator model file in Generating the IVFSP Training Operator Model File. The value must be greater than 0. The default value is 32768.

int codeNum: maximum number of codebooks to be operated. The value must be a power of 2. The value of this parameter must be the same as that of <codebook_batch_size> for generating the IVFSP training operator model file in Generating the IVFSP Training Operator Model File. The value must be greater than 0. The default value is 32768.

std::string codeBookOutputDir: directory to which the generated codebook file is output. Ensure the target directory exists and the program running user is granted write permissions. For security hardening, this directory level cannot contain soft links.

bool verbose: whether to print extra information. The default value is true.

const float *memLearnData: data pointer in the memory. The default value is a null pointer.

size_t memLearnDataSize: data length in the memory. The default value is 0.

bool isTrainAndAdd: whether to train the codebook and add the trained codebook to the index. The default value is false.

• numIter ∈ (0, 20]
• ratio ∈ (0, 1.0]
• memLearnDataSize % dim == 0
• memLearnDataSize ≤ 25 GB

• When a codebook file exists, it will be overwritten. In this case, the program running user should be the owner of the file.
• Before executing the generated training codebook, generate a training operator model file by referring to IVFSP.

• When isTrainAndAdd is set to true, the codebook is directly added to the index after being trained and is not flushed to the drive.
• When isTrainAndAdd is set to false, the codebook is saved to the path specified by codeBookOutputDir and needs to be manually added by calling addCodeBook.
• memLearnDataSize must be the actual length of memLearnData. Otherwise, an out-of-bounds read/write error may occur, causing program breakdown.