Overview

Background

In recent years, the evolution of artificial intelligence (AI) technologies has enabled sophisticated algorithms and models to proficiently extract feature representations from unstructured data types such as images, texts, and voices. These representations can be viewed as structured vector features. In real-world applications, the ability to quickly and accurately locate a vector that resembles a to-be-queried vector has emerged as a crucial need for various intelligent applications. Consequently, there is a demand for an efficient vector feature retrieval system, with a rapid retrieval engine as its core.

In response to this need, Huawei provides an efficient vector feature retrieval engine that is built on Index SDK of the Ascend platform. This allows users to execute retrievals oriented to application scenarios on this engine.

Product Definition

FeatureRetrieval is an Ascend NPU heterogeneous retrieval acceleration framework that is built on Faiss, providing high-performance retrieval for massive data in high-dimensional space. It is developed based on TBE operators and the C++ language that is consistent with Faiss and supports Arm and x86_64 platforms. FeatureRetrieval supports two types of retrieval libraries: small database retrieval (full retrieval) and large database retrieval (approximate retrieval). A small database contains 300,000 to 1,000,000 records, while a large database contains tens of millions or even hundreds of millions of records and supports feature vectors with 64 dimensions to 512 dimensions. (Dimensions vary according to algorithms.)

• The small databases retrieval (full retrieval) mainly uses brute-force search algorithms such as Flat, SQ, and INT8. Full retrieval is performed for feature vectors in a database, and top-K distance sorting results are returned.
  • The INT8 algorithm performs brute-force search based on feature quantization, which also refers to int8flat in this document (for example, int8flat_generate_model.py).
  • The SQ algorithm uses an 8-bit integer for quantization, which also refers to SQ8 in this document (for example, sq8_generate_model.py).
• For the large database retrieval (approximate retrieval), the IVFSQ algorithm is executed on the Ascend platform based on the Faiss framework. Different from the conventional inverted indexing, the IVF-based algorithm first performs feature clustering and then reduces the retrieval scale using cluster centroids, improving performance at the cost of accuracy.

The low level of each algorithm is implemented by TBE operators accelerated by the Ascend platform.

In addition, FeatureRetrieval supports attribute filtering-based retrieval and multi-index batch retrieval.

• For Attribute-based filtering, time- and space-related attributes are added when adding vectors. Then, vectors under specific time and spatial conditions can be used for retrieval.
• Multi-index batch retrieval allows users to use multiple indexes to partition a database and use a unified interface to retrieve multiple indexes at once.

Product Benefits

As a high-performance vector retrieval SDK, Index SDK has the following benefits:

• Compatibility with mainstream frameworks: native Faiss APIs provide out-of-the-box usage.
• High performance: The full retrieval performance of cards with equivalent computing power surpasses industry standards, and the batch retrieval performance outperforms that of serial scenarios.
• Large capacity: Supports vector retrieval in a database with hundreds of millions of data records on a single card.