Introduction

Overview

The operators provided here are built-in CANN operators available for direct calls. For details about how to develop custom operators, see Ascend C Operator Development.
You can find the CANN operator source code in CANN on GitCode, where you can learn more about the operator development and contribution process.

Compute Architecture for Neural Networks (CANN) provides a diverse array of hardware-adaptive operators, directly applicable to AI services and laying a foundation for accelerating network computing on AI processors.

Figure 1 shows where the operator libraries sit in the CANN architecture:

Math: numerical calculation operators, like Add and Abs, for use cases such as tensor shape transformation, basic mathematical operations, and random number generation.
NN: neural network operators, common in deep learning models, for use cases such as convolution, matrix multiplication, activation functions, and normalization.
CV: computer vision operators, such as GridSample, for image processing and object detection.
Transformer: foundation model computation operators in major classes such as Attention, LayerNorm, and merged compute and communication (MC2).
RAS: reliability, availability, and serviceability (RAS) operators.
Opbase: basic framework library on which all other operator libraries depend. It realizes basic scheduling (such as aclTensor creation/release and workspace reuse) and common dependencies.

Figure 1 Operator library architecture

CANN provides multiple methods to call operators. Select a method as needed.

aclnn APIs: A set of C APIs is provided for all operators to facilitate calling on the host. These single-operator APIs (or operator host APIs) are called as shown in Figure 2.
PyTorch APIs: A set of torch_extension APIs compatible with the native PyTorch is provided for common large model operators that are not native PyTorch operators. These APIs use the Just-In-Time (JIT) (torch.utils.cpp_extension.load) mechanism to compile the C++ Kernel Wrapper in real time upon the first call and bridges PyTorch functions to aclnn APIs. In addition, it supports the TorchAir graph mode through GE Converter.
GE graph mode: Operators are called by defining Ascend Intermediate Representations (IRs). For details about the calling, see Programming Guide in Graph Development.

Figure 2 aclnn API calling process

Instructions

Version evolution: An operator or operator API may have multiple versions. Select the latest version (later versions are compatible with earlier versions by default).
CANN does not guarantee operator performance in use cases beyond the support scope in operator documentation (such as product models, data types, data formats, and data dimensions).
Ascend virtual instance: Not supported for fused operators in the current version. A fused operator, such as Flash Attention or MC2, is a combination of multiple independent basic operators (such as vector and cube). It delivers the same functions as those basic operators but with better performance.

Instructions

Before reading the following sections, familiarize yourself with this wizard to understand the operator classification and use cases.

**Table 1** Instructions
Use Case	Guide	Description
Dependent header files/library files	Header Files and Library Files	How to call the header files or library files on which operators depend in different ways
Operator library basics	Basic Concepts	Basic concepts, terminology, and key technologies of operators
Dependent common APIs for operator API calling	Common APIs	Common meta APIs such as those for creating aclTensor, aclScalar, and aclIntArray
aclnn APIs in operator libraries	Math APIs	Mathematical calculation operator library, providing C APIs for operators such as Add and Abs
	NN APIs	Neural network operator library, providing C APIs for operators such as Matmul Currently, operators of this type account for the largest proportion in the operator library.
	CV APIs	Computer vision operator library, providing C APIs for operators such as GridSample
	Transformer APIs	Large model computation operator library, providing C APIs for operators and use cases such as FlashAttention, MC2 (merged compute and communication), and Mixture of Experts (MoE).
	RAS APIs	Reliability, availability, and serviceability (RAS) operators.
Torch extension APIs in the operator library	Torch extension APIs	A set of PyTorch APIs for common large model operators that are not native PyTorch operators. The C++ Kernel Wrapper is JIT compiled to bridge PyTorch functions to aclnn APIs. In addition, the GE Converter is used to support the TorchAir graph mode.
Ascend IR operator specifications in the operator library	Ascend IR Operator Specifications	Operator information defined based on the Ascend IR.
Operator library knowledge	Appendix	nnopbase APIs on which the aclnn API development depends op_common APIs on which operator development depends Operator performance improvement methods FAQ and cases for operator calls