Description of feature vector search
Deep learning algorithms are applied in image feature searching to process each region of interest (ROI) and extract a series of high-dimensional feature vectors. The extracted feature vectors are compared against every feature vector in an existing repository to find the most similar ones. To further reduce memory footprint and compute overheads, the long feature vectors in floating-point formats are usually quantized to short feature vectors before feature comparison.
The AI processor enables hardware-accelerated PQ short feature search and provides APIs for adding/deleting feature repositories, modifying/deleting feature vectors, and performing feature search. It supports a short feature vector of 32 bytes, a configurable number of top results to return, and the 1:N and N:M search modes. You need to prepare a short feature repository and generate ADC tables of the vectors to be searched. The following figure illustrates the basic workflow.

The following figure shows the search and comparison process.

The ADC table is a 32 KB lookup table, whose content needs to be passed to the corresponding API calls in the format of unsigned characters.
The following table describes the feature search specifications of the
|
Specification |
Description |
|---|---|
|
Feature length |
32 bytes (The average memory overhead of each feature is about 36 bytes.) |
|
Repository capacity |
Up to 500 million features (depending on the memory capacity of the board) |
|
Search modes |
1:N mode and N:M mode |
|
Number of repositories |
1:N mode: up to 1 million repositories N:M mode: up to 1 repository |
|
Repository capacity |
1:N mode: up to 1 million features N:M mode: up to 10 million features |
|
Search and comparison performance |
5 billion queries per second (QPS) 10 QPS in 500 million features with less than 1s latency |
|
Number of top results to return |
2–4800 |
API Call Process
Feature vector search is used to verify the feature search function. To do so, you need to generate a random repository in which random features can be searched for. (Currently, the 1:N and M:N search modes are supported. The following sample code uses the 1:N mode as an example.) The process can be broken into the following steps: initialization, adding features to the repository, repository search, precise modification or deletion of the features in the repository, and deinitialization. The API calls are described as follows:
- Initialization: Call aclInit for initialization and aclfvCreateInitPara to create data of the aclfvInitPara type to specify the initialization parameters for feature vector search.
- Add a feature to the base library: Call aclfvCreateFeatureInfo to create data of the aclfvFeatureInfo type to describe the feature, and then call aclfvRepoAdd to add a base library.
- Base library search: The aclfvSearch API is called to implement search.
- Precisely modifying or deleting a base library feature: Call the aclfvDel and aclfvModify APIs to delete or modify a feature in the base library. The following code uses feature deletion as an example.
- Deinitialization: releases runtime resources, calls the aclfvDestroyInitPara API to destroy data of the aclfvInitPara type, and calls the aclfvRelease API to deinitialize the feature retrieval module and release memory space.
Sample Code
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 |
...... // 1. Perform initialization. // 1.1 Initialize the feature retrieval module. The following uses the number of features in the base library (100000) as an example. size_t fsNum = 100000; fvInitPara = aclfvCreateInitPara(fsNum); // 1.2 Specify the initialization parameters for feature vector search. ret = aclfvInit(fvInitPara); // 2. Add the base library and feature vectors. // 2.1 Add the first feature. When creating the feature description, set the offset parameter to 0. uint32_t offset = 0; uint32_t featureCount = 1000; uint32_t featureLen = 36; //The user-defined function BaseShortFeaAlloc is used to generate random feature data. void *featureData = BaseShortFeaAlloc(1000, static_cast<size_t>(featureCount), 0); std::shared_ptr<void> feaBufPtr(featureData, [](void *p){(void)aclrtFreeHost(p);}); void *inputData = featureData; std::shared_ptr<void> inputDataPtr = nullptr; // If the run mode is ACL_HOST, allocate memory and call aclrtMemcpy to transfer the random feature data of the host to the device. Otherwise, directly read the random feature data into the device memory. if (aclrtGetRunMode(&runMode) == ACL_HOST) { //Allocate memory for input data. ret = aclrtMalloc(&inputData, featureLen * featureCount, ACL_MEM_MALLOC_HUGE_FIRST); //Read the random feature data to the device memory. inputDataPtr.reset(inputData, [](void *p) {(void)aclrtFree(p);}); //Copy the feature data from the host to the device. ret = aclrtMemcpy(inputData, featureLen * featureCount, featureData, featureLen * featureCount, ACL_MEMCPY_HOST_TO_DEVICE); } // Create a feature description. inputData indicates the random feature data in the previous step. auto featureInfo = aclfvCreateFeatureInfo(id0, id1, offset, featureLen, featureCount, reinterpret_cast<uint8_t *>(inputData), featureLen * featureCount); // Add a base library and add features to the base library. featureInfo indicates the feature description in the previous step. aclError ret = aclfvRepoAdd(SEARCH_1_N, featureInfo); // Destroy the aclfvFeatureInfo feature description. aclfvDestroyFeatureInfo(featureInfo); // 2.2 Add the second feature. When creating the feature description information, ensure that the offset value is the same as the number of added features in the library and precisely delete or modify a feature in the base library. offset += featureCount; //For details about how to add a feature to the repository, see step 4.1. // .... uint8_t featureData[36]; for (size_t i = 0; i < 36; i++) { featureData[i] = static_cast<uint8_t>(i); } // Create memory and transfer feature data. void *inputData = nullptr; aclrtMalloc(&inputData, 36, ACL_MEM_MALLOC_HUGE_FIRST); std::shared_ptr<void> inputDataPtr(inputData, [](void *p){(void)aclrtFree(p);}); aclrtMemcpyKind kind = ACL_MEMCPY_DEVICE_TO_DEVICE; // If the running mode is ACL_HOST, copy the feature data to the device. Otherwise, you do not need to copy the feature data. dataLen indicates the length of the memory allocated by the featureData pointer. if (aclrtGetRunMode(&runMode) == ACL_HOST) { kind = ACL_MEMCPY_HOST_TO_DEVICE; } aclrtMemcpy(inputData, 36, featureData, dataLen, kind); // Create feature description information. uint32_t id0 = 0; uint32_t id1 = 0; auto featureInfo1 = aclfvCreateFeatureInfo(id0, id1, offset, 36, 1, reinterpret_cast<uint8_t *>(inputData), 36); std::shared_ptr<aclfvFeatureInfo> featureInfoPtr(featureInfo1, [](aclfvFeatureInfo *p){(void)aclfvDestroyFeatureInfo(p);}); //Delete a feature. aclfvDel(featureInfo1); // 2.3 Add features to other base libraries. The value is 1 for the level-1 base library and 1 for the level-2 base library. id0 = 1; id1 = 1; offset = 0; //For details about how to add a feature to the repository, see step 4.1. // .... // 3 Base library search (1:N search is used as an example), including feature search preprocessing, 1:N feature search, and feature search result processing. // 3.1 Feature retrieval preprocessing. For 1:N, queryCnt must be 1. uint32_t queryCnt = 1; uint32_t topK = 5; uint32_t dataLen = queryCnt * topK * sizeof(uint32_t); uint32_t resultNumDataLen = queryCnt * sizeof(uint32_t); const uint32_t tableLen = 32 * 1024; uint32_t tableDataLen = queryCnt * tableLen; // Generate a data table. You can search for and compare data based on the data table. The user-defined function AdcTabInit is used to initialize the input ADC table for feature retrieval. uint8_t *tableDataTmp = (uint8_t *)AdcTabInit(1000, queryCnt * 1024); std::shared_ptr<void> tableDataTmpPtr(tableDataTmp,[](void *p){(void)aclrtFreeHost(p);}); // Allocate memory for the data table. tableDataDev is used to create the retrieval input table information. void *devPtr = nullptr; aclrtMalloc(&devPtr, tableDataLen, ACL_MEM_MALLOC_HUGE_FIRST); tableDataDev.reset(devPtr, [](void *p) {(void)aclrtFree(p);}); // Copy the table data to the device. uint8_t *devPtrTmp = reinterpret_cast<uint8_t *>(devPtr); for (uint32_t i = 0; i < queryCnt; ++i) { for (uint32_t j = 0; j < 32; ++j) { uint8_t *dst = devPtrTmp + i * 32 * 1024 + j * 1024; uint8_t *src = tableDataTmp + i * 1024; aclrtMemcpy(dst, 1024, src, 1024, ACL_MEMCPY_HOST_TO_DEVICE); } } //Allocate memory for the search result resultNumDev, id0Dev, id1Dev, resultOffsetDev, and resultDistanceDev. aclrtMalloc(&devPtr, resultNumDataLen, ACL_MEM_MALLOC_HUGE_FIRST); resultNumDev.reset(devPtr, [](void *p) {(void)aclrtFree(p);}); aclrtMalloc(&devPtr, dataLen, ACL_MEM_MALLOC_HUGE_FIRST); id0Dev.reset(devPtr, [](void *p) {(void)aclrtFree(p);}); aclrtMalloc(&devPtr, dataLen, ACL_MEM_MALLOC_HUGE_FIRST); id1Dev.reset(devPtr, [](void *p) {(void)aclrtFree(p);}); aclrtMalloc(&devPtr, dataLen, ACL_MEM_MALLOC_HUGE_FIRST); resultOffsetDev.reset(devPtr, [](void *p) {(void)aclrtFree(p);}); aclrtMalloc(&devPtr, dataLen, ACL_MEM_MALLOC_HUGE_FIRST); resultDistanceDev.reset(devPtr, [](void *p) {(void)aclrtFree(p);}); //Create a search input table. The result is used as the input information for creating a search task. aclfvQueryTable *searchQueryTable = aclfvCreateQueryTable(queryCnt, tableLen, reinterpret_cast<uint8_t *> (tableDataDev.get()), tableDataLen); searchQueryTable.reset(searchQueryTable, [](aclfvQueryTable *p){(void)aclfvDestroyQueryTable(p);}); //Create a feature repository range parameter. The result is used as the input information for creating a search task. aclfvRepoRange *searchRange = aclfvCreateRepoRange(0, 1023, 0, 1023); searchRange.reset(searchRange, [](aclfvRepoRange *p){(void)aclfvDestroyRepoRange(p);}); // Create a search task. The result is used for 1:N feature search. aclfvSearchInput *searchInput = aclfvCreateSearchInput(searchQueryTable, searchRange, topK); searchInput.reset(searchInput, [](aclfvSearchInput *p){(void)aclfvDestroySearchInput(p);}); // Create a search result for 1:N feature search. aclfvSearchResult *searchResult = aclfvCreateSearchResult(queryCnt, reinterpret_cast<uint32_t *>(resultNumDev.get()), resultNumDataLen, reinterpret_cast<uint32_t *>(id0Dev.get()), reinterpret_cast<uint32_t *>(id1Dev.get()), reinterpret_cast<uint32_t *>(resultOffsetDev.get()), reinterpret_cast<float *>(resultDistanceDev.get()), dataLen); searchResult.reset(searchResult, [](aclfvSearchResult *p){(void)aclfvDestroySearchResult(p);}); // 3.2 1:N feature search aclfvSearch(SEARCH_1_N, searchInput.get(), searchResult.get()); // 3.3 Process the feature retrieval result. // Obtain the search result. uint32_t dataLen = queryCnt * topK * sizeof(uint32_t); uint32_t *id0 = (uint32_t *)id0Dev.get(); uint32_t *id1 = (uint32_t *)id1Dev.get(); uint32_t *resultOffset= (uint32_t *)resultOffsetDev.get(); float *resultDistance = (float *)resultDistanceDev.get(); //If the run mode is ACL_HOST, the search result on the device needs to be sent back to the host through the aclrtMemcpy call. if (aclrtGetRunMode(&runMode) == ACL_HOST) { // Copy data from the device to the host. id0 = (uint32_t *)malloc(dataLen); id0Ptr.reset(id0); id1 = (uint32_t *)malloc(dataLen); id1Ptr.reset(id1); resultOffset = (uint32_t *)malloc(dataLen); resultOffsetPtr.reset(resultOffset); resultDistance = (float *)malloc(dataLen); resultDistancePtr.reset(resultDistance); aclrtMemcpy(id0, dataLen, id0Dev.get(), dataLen, ACL_MEMCPY_DEVICE_TO_HOST); aclrtMemcpy(id1, dataLen, id0Dev.get(), dataLen, ACL_MEMCPY_DEVICE_TO_HOST); aclrtMemcpy(resultOffset, dataLen, resultOffsetDev.get(), dataLen, ACL_MEMCPY_DEVICE_TO_HOST); aclrtMemcpy(resultDistance, dataLen, resultDistanceDev.get(), dataLen, ACL_MEMCPY_DEVICE_TO_HOST); } // Display data in the base library. for (uint32_t i = 0; i < queryCnt; i++) { for (uint32_t j = 0; j < topK; ++j) { uint32_t i0 = id0[i * topK + j]; uint32_t i1 = id1[i * topK + j]; uint32_t offset = resultOffset[i * topK + j]; float distance = resultDistance[i * topK + j]; } } // 4. Delete the base library and data. //Create a feature repository range and delete the repository in the specified range. uint32_t id0Min = 0; uint32_t id0Max = 1023; uint32_t id1Min = 0; uint32_t id1Max = 1023; aclfvRepoRange *repoRange = aclfvCreateRepoRange(id0Min, id0Max, id1Min, id1Max); aclfvRepoDel(SEARCH_1_N, repoRange); //Destroy data of the aclfvInitPara type. aclfvDestroyInitPara(fvInitPara); ...... |