SearchWithExtraMask (with Extra Similarity)

API definition

APP_ERROR SearchWithExtraMask(uint32_t count, const void *features, const AttrFilter *attrFilter, bool shareAttrFilter, uint32_t topk,const uint8_t *extraMask, uint64_t extraMaskLenEachQuery, bool extraMaskIsAtDevice, const uint16_t *extraScore, int64_t *labels,float *distances, uint32_t *validNums, bool enableTimeFilter = true);

Function

Calculates the distance between the input feature and the base vector filtered by AttrFilter and the external mask, sorts the distance by TopK, and returns the corresponding distance and index.

Input

uint32_t count: number of features to be compared.

const void *features: feature to be compared. The Hamming distance requires data of the uint8_t type, Int8Flat requires data of the int8_t type, and FP16Flat requires data of the float type.

const AttrFilter *attrFilter: attribute filtering information. For details, see SearchWithExtraMask.

bool shareAttrFilter: whether a query shares a mask.

uint32_t topk: size of the TopK feature to be saved after the cosine distance is calculated.

const uint8_t *extraMask: extra filtering mask input externally, in bits. The values 0 and 1 indicate that the feature is filtered or selected, respectively.

uint64_t extraMaskLenEachQuery: length of the externally input mask, in bytes.

bool extraMaskIsAtDevice: whether the externally input mask exists on the device.

const uint16_t *extraScore: extra similarity input by the user. The length is count × totalPad (16-aligned database size).

bool enableTimeFilter: whether to filter the timestamp attribute. The default value is true. If enableTimeFilter is set to false, the timestamp attribute is not filtered.

Output

int64_t *labels: labelof the TopK feature. If features are added by calling AddFeatureByIndice, indices is output.

float *distances: distance of the TopK feature.

uint32_t *validNums: number of valid results obtained after each query vector is compared.

Return value

APP_ERROR: return status. For details, see Return Code Reference.

Restrictions

  • Value range of count: [1, 10240]
  • Value range of topk: [1, 100000]
  • The length of features must be count × dim (vector dimension). Otherwise, an out-of-bounds read/write error may occur, causing program breakdown.
  • attrFilter: When shareAttrFilter is true, the length is 1. When shareAttrFilter is false, the length is count. Otherwise, an out-of-bounds read/write error may occur, causing program breakdown.
  • The length of distances must be count × topk. Otherwise, an out-of-bounds read/write error may occur, causing program breakdown.
  • The length of validNums must be count. Otherwise, an out-of-bounds read/write error may occur, causing program breakdown.
  • The length of labels must be count × topk. Otherwise, an out-of-bounds read/write error may occur, causing program breakdown.
  • extraMask: When shareAttrFilter is true, the length is extraMaskLenEachQuery. When shareAttrFilter is false, the length is count × extraMaskLenEachQuery. Otherwise, an out-of-bounds read/write error may occur, causing program breakdown.
  • extraScore: The length is count × totalPad (16-aligned database size). Otherwise, an out-of-bounds read/write error may occur, causing program breakdown. The value is of the float16_t type and ranges from -1.0 to 1.0. Currently, this parameter is valid only for Int8FlatCos and FlatIP both with non-shared mask. Otherwise, extraScore is not involved in the calculation.