Feature Extraction

Function Description

Feature extraction can be performed on the input image by constructing the Sift class instance. Specify an input image tensor and the mask rectangle used to limit the feature extraction area, call the feature extraction API to perform model inference, and output the extracted feature point list and descriptor list.

Before using the Sift class to extract feature points, generate an OM model for building scale space in advance. Perform the following steps:

1. Set the environment variables of the CANN development kit. Here, ${CANN_INSTALL_PATH} is set to the default path ($HOME) of a common user.

   1	. ${CANN_INSTALL_PATH}/cann/set_env.sh

2. Set the environment variables of Vision SDK, where ${MX_SDK_HOME} indicates the installation directory of Vision SDK.

   1	. ${MX_SDK_HOME}/set_env.sh

3. Access the installation directory of Vision SDK.

   1	cd ${MX_SDK_HOME}

4. Create and go to the data directory for storing the model weight file.

   1 2	mkdir data cd data

5. Execute generate_sift_weights.py in the ${MX_SDK_HOME}/bin directory to create the weight file required by the model.

   1	python3 ../bin/generate_sift_weights.py

6. Access the bin file in the installation directory of Vision SDK.

   1	cd ${MX_SDK_HOME}/bin

7. Run the executable file sift to create an OM model. The chip version parameter soc_version can be input.
8. The chip version corresponding to Atlas 200I A2 accelerator module (20 TOPS, 12 GB) is Ascend310B*. The asterisk (*) represents different values according to factors such as the chip performance improvement level and core usage level. Run npu-smi info to obtain details.

   ./sift soc_version

API Calling Process

Figure 1 Process of calling APIs of feature extraction

1. Perform global initialization by calling MxInit().
2. Before calling the feature extraction APIs, initialize ImageProcessor and call the image decoding API of ImageProcessor to obtain the input Image. For details, see Image Decoding.
3. Call the ConvertToTensor API of the Image class to convert the input Image into a tensor.
4. Feature extraction of Sift supports only single-channel images. Therefore, call the CvtColor API of the tensor class to convert the tensor into a single-channel grayscale image tensor in the HWC format. For details, see CSC.
5. Define the input mask rectangle, which is used to limit the area for feature calculation. Perform feature extraction on the image in the area. The coordinates of the upper left and lower right corners in the rectangle must be within the valid range of the image.
6. Define the feature point list and descriptor list. You can assign values to the feature point list. The APIs directly generate descriptors based on the user-defined feature points.
7. Construct the feature extraction class and initialize the model extraction resources.

   To adjust parameter values when constructing the feature extraction class, pay attention to exceptions. You are advised to use try/catch to capture exceptions.

8. Call the feature extraction APIs to perform calculation.
9. Deinitialize the initialized global resources by calling MxDeInit().

Sample Code

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56

// Initialization
MxBase::MxInit();

{
    // Construct the image processing class.
    MxBase::ImageProcessor imageProcessor(deviceId);

    // Decode the image for Image generation.
    // Decoded image class
    MxBase::Image image;

    // Perform decoding based on the image path.
    APP_ERROR ret = imageProcessor.Decode(imagePath, image);
    if (ret != APP_ERR_OK) {
        return 0;
    }

    // Convert the image into a tensor and set the tensor layout format to HWC.
    MxBase::Tensor tensor = image.ConvertToTensor(true, false);

    // Perform CSC on the image using the tensor method.
    // Define the CSC type.
    auto mode = MxBase::CvtColorMode::COLOR_YUVSP4202GRAY;

    // Define the output tensor.
    MxBase::Tensor imgTensor;

    // Perform CSC.
    MxBase::CvtColor(tensor, imgTensor, mode, true);
 
    // Define the mask rectangle.
    MxBase::Rect mask = MxBase::Rect(x0, x1, y0, y1);
 
    // Define the feature point list.
    vector<cv::KeyPoint> keyPoints;
 
    // Define the descriptor list.
    cv::Mat descriptors;

    try {
       // Construct the feature extraction class.
        Sift sift(nFeatures, nOctaveLayers, contrastThreshold, edgeThreshold, sigma, descriptorType);
 
        // Initialize model feature extraction resources.
        sift.Init(deviceId);
 
        // Perform feature extraction.
        sift.DetectAndCompute(imgTensor, mask, keyPoints, descriptors, false);
    } 
    catch (runtime_error error) {
        return 0;
    }
}

// Deinitialization
MxBase::MxDeInit();