Overview

In the inference service, some operators (such as where) in the recommendation model have poor affinity on the NPU and may be split to the CPU for computing. Therefore, the SVE instruction set is used to optimize the performance of TensorFlow CPU operators. For details about SVE, see Introduction to SVE and ARM C Language Extensions for SVE.

This section describes how to use the SVE instruction set to optimize four operators on the TensorFlow CPU side: less, greater, floormod, and where.

You can obtain the component source code of Rec SDK TensorFlow. For details about how to install and use the component, see the cust_op/tf_cpu_op/README.md file in the source code.