Introduction

Overview

The Compute Architecture for Neural Networks (CANN) operator library provides a series of high-performance operators, with enriched, in-depth optimizations and hardware-affinity features, laying a foundation for accelerating network computing on AI processors.

Figure 1 shows the position of the operator library in the CANN architecture. The operator library consists of the following sub-libraries:

Math library: Mathematical calculation operator library, which provides operators like Add and Abs.
NN library: Neural Network, that is, neural network operator library, which provides operators like Matmul.
CV library: Computer Vision, that is, computer vision operator library, which provides operators like GridSample.
Transformer library: Large model computing operator library, which provides operators like FlashAttention, MC2 (merged compute and communication), and mixture of experts (MoE).
Opbase library: Basic framework layer on which each operator library depends. It provides basic scheduling capabilities (such as aclTensor creation/release and workspace reuse) and common dependency capabilities.

The operators provided by this operator library are built-in operators. For details about how to customize operators, see Ascend C Operator Development Guide.
The source code of certain operators in this operator library has been opened in the Gitcode CANN open-source project. Click Link to learn more about the component project of the operator library.

Figure 1 Operator library architecture

To facilitate the calling of the preceding operators, a set of C-based APIs (such as aclnn APIs) are provided to execute operators, without the need to provide additional Intermediate Representation (IR) definitions. This calling mode is called single-operator API execution. Figure 2 shows the calling process. For details, see "Single-Operator Call Sequence".

Figure 2 Operator API calling process

This document describes operator specifications and operator APIs, and details the operator specifications defined by IR in different AI deep learning frameworks (such as TensorFlow and Caffe), helping developers build required network models.

Instructions

You are advised not to use scenarios beyond the operator APIs or operator specifications (such as product models, data types, data formats, and data dimensions). This is because the calling effect is not guaranteed in the current version.
Operator package description: Before calling an operator, ensure that software packages for the CANN Toolkit and OPS operator have been installed. For details, see .
Operator troubleshooting: During operator calling, various exceptions may occur. For details about how to handle the exceptions, see Troubleshooting Cases > Operator Execution Issues. The document describes typical and frequently occurred operator executions for you to locate and resolve issues.
Ascend virtual instance: The current fusion operator does not support this function. A fusion operator, such as Flash Attention or MC2, is a combination of multiple independent basic operators (such as vector and matrix cube). It has equivalent functions as multiple basic operators and delivers better performance.

Usage Guide

Before reading this document, you are advised to get yourself familiarized with Table 1, which helps you quickly understand the application scenarios, operator classification, and operator APIs of the operator library.

Header file and library file descriptions: To call an operator API, include the dependent files. Generally, header files are stored in the ${INSTALL_DIR}/include/ directory by default, and library files are stored in the ${INSTALL_DIR}/lib64/ directory by default. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

**Table 1** Usage Guide
Use Case	Guidance	Description	Dependent Header File or Library File
1. Basic knowledge that users need to learn before using the operator library	Basic Concepts	This section describes the basic concepts, terms, and key technologies related to the operator library, helping you better understand the operator library.	-
2. Common APIs on which the operator library depends during API calling	Common APIs	Declares common meta APIs that the operator APIs depend on, such as the APIs for creating aclTensor, aclScalar, and aclIntArray.	Header files: aclnn/acl_meta.h and aclnn/aclnn_base.h (containing only aclnnInit and aclnnFinalize) Library file: libnnopbase.so
3. Types of operator APIs (aclnn)	Math APIs	Mathematical calculation operator library, which provides operators like Add and Abs. These APIs are prefixed with aclnnXxx.	Header file: After the general header file of each type is referenced, the header file of a single API does not need to be referenced. (Recommended) General header files on which various types of APIs depend: aclnnop/aclnn_ops_math.h, aclnnop/aclnn_ops_nn.h, aclnnop/aclnn_ops_cv.h, and aclnnop/aclnn_ops_transformer.h Header file on which each API depends: aclnnop/aclnn_.h ( indicates the operator name.) Dynamic library: After the general library file is referenced, the library file of a single API does not need to be referenced. The two types of library files cannot both be referenced. Library file on which the aclnn API depends: libopapi.so (Recommended) Library files on which various types of APIs depend: libopapi_math.so, libopapi_nn.so, libopapi_cv.so, and libopapi_transformer.so NOTE: In CANN 8.5.0 and later versions, the static libraries (.a) and dynamic libraries (.so) libaclnn_ops_infer, libaclnn_ops_train, libaclnn_math, and libaclnn_rand have been deprecated. Do not use them. Use the latest dynamic library file with the libopapi prefix. In CANN 8.5.0 and later versions, you are advised to reference the .so sub-library file corresponding to each type of API to improve operator compilation efficiency. You are advised not to use the entire library libopapi.so. In CANN 7.0.0 and later versions, the operator header file path *aclnnop/level2/aclnn_.h is not recommended. Use the new header file path aclnnop/aclnn_.h* instead.
	NN APIs	Neural network (NN) operator library, which provides operators like Matmul. These APIs are prefixed with aclnnXxx. Currently, this type of operators accounts for the largest proportion in the operator library.
	CV APIs	Computer vision (CV) operator library, which provides operators like GridSample. These APIs are prefixed with aclnnXxx.
	Transformer APIs	Large model computing operator library, which provides operators like FlashAttention, MC2 (merged compute and communication), and mixture of experts (MoE). These APIs are prefixed with aclnnXxx.
4. Media data processing APIs (acldvpp)	Media Data Processing APIs (acldvpp)	Digital vision preprocessing operators, which provide preprocessing APIs for high-performance video/image encoding and decoding, as well as image cropping and resizing. These APIs are prefixed with acldvppXxx.	Header files: acldvppop/acldvpp_base.h and acldvppop/acldvpp_op_api.h Library file: libacl_dvpp_op.so
5. Operator information defined based on the Ascend IR in the operator library	Ascend IR Operator Specifications	Operator information defined based on the Ascend IR.	-
6. Operator information defined based on the native IR of the AI framework	Operators Specifications of the AI Framework	Operator information defined based on the native IR of mainstream AI frameworks (such as TensorFlow and Caffe).	-
7. Operator library APIs	Appendix	Framework APIs on which the aclnn API for developing built-in operators depends. For details, see aclnn Development APIs.	-