Header Files and Library Files
API Category
The ATB library provides C++ and C APIs for inference and training. APIs are classified into the following types by function:
API Type |
Naming Convention |
Description |
|---|---|---|
Basic types and conventions |
atb:: - namespace, without a special prefix |
Basic data structures (TensorDesc, Dims, and VariantPack), error codes (Status), utility functions (Utils), and containers (SVector) |
Context management |
atb::Context / CreateContext / DestroyContext |
Runtime context management, including stream settings, tiling attributes, and startup modes (single operator or graph) |
Operator definition and execution |
atb::Operation / CreateOperation / DestroyOperation |
Abstract base class of operators, which provides lifecycle APIs such as Setup (precomputing the workspace size) and Execute (executing an operator). |
Graph building |
atb::GraphOpBuilder / CreateGraphOpBuilder |
Combines multiple operators into a computational graph and supports graph operations such as Reshape, AddOperation, and Build. |
Inference operator parameters |
atb::infer::*Param |
Parameter structs of inference operators, such as LinearParam, SelfAttentionParam, and RmsNormParam |
Training operator parameters |
atb::train::*Param |
Parameter structs of training operators, such as FastSoftMaxParam, LaserAttentionParam, and RopeGradParam |
Communicator APIs |
atb::Comm:: - prefix |
HCCL communicator management, including communicator creation and destruction |
Operator APIs in C |
atb - prefix, two-phase calling |
C APIs for specific heterogeneous operators, such as AtbMLA, AtbFusedAddTopkDiv, and AtbRingMLA |
Communication operator parameters |
atb::common::*Param |
Such as Event synchronization (EventParam) and conditional branch (IfCondParam) |
Plug-in extension base class |
atb::OperationInfra |
Infrastructure class for custom/plug-in operators, inherited from atb::Operation |
Dependent Header Files and Library Files
After installing the ATB, reference the header files and library files of ATB APIs when building and running applications.
The header files are stored in the ${ATB_HOME_PATH}/include/atb/ directory, and the library files are stored in the ${ATB_HOME_PATH}/lib/ directory. ${ATB_HOME_PATH} indicates the ATB installation path, typically set via source <Installation directory>/atb/set_env.sh.
For example, if the installation is performed as user root, the file storage paths after the installation are as follows:
- Header file: /usr/local/Ascend/atb/cxx_abi_1/include/atb/ or /usr/local/Ascend/atb/cxx_abi_0/include/atb/
- Library file: /usr/local/Ascend/atb/cxx_abi_1/lib/ or /usr/local/Ascend/atb/cxx_abi_0/lib/
The value of cxx_abi depends on the C++ ABI version of the compilation environment. If PyTorch in the environment is compiled using CXX11 ABI, use cxx_abi_1. Otherwise, use cxx_abi_0. You can use source atb/set_env.sh --cxx_abi=1 or --cxx_abi=0 to specify it.
When compiling the ATB API program, include the header files to depend on the matching library. If you require only the inference function, reference libatb.so only. If you need the training operator, reference libatb_train.so additionally. Referencing unnecessary .so files may cause version function exceptions or compatibility issues during subsequent version upgrades.
Include the dependent header files based on the ATB APIs to be called. You are advised to include atb/atb_infer.h as the general entry header file. This file contains all references to external header files. The following table lists the purpose and the matching library of each header file.
Header File |
Purpose |
Matching Library |
|---|---|---|
atb/atb_infer.h |
Main entry header file, which contains the references to all external header files |
libatb.so |
atb/types.h |
Definition of basic data structures, including Status (error code), Dims (dimension description), TensorDesc (tensor descriptor), Tensor (tensor data), and VariantPack (input and output tensor package) |
libatb.so |
atb/svector.h |
Definition of the vector container SVector<T> on the stack |
libatb.so |
atb/context.h |
Context API class, used to set the operator execution context, such as the execution stream, tiling attributes, and startup mode, as well as the CreateContext/DestroyContext factory functions |
libatb.so |
atb/operation.h |
Abstract base class of the Operation operators, which provide operator lifecycle APIs (Setup/Execute), shape inference (InferShape), and factory functions such as CreateOperation/DestroyOperation |
libatb.so |
atb/operation_infra.h |
Operator infrastructure class of the OperationInfra plug-in, inherited from Operation |
libatb.so |
atb/infer_op_params.h |
Definition of inference operator parameters, including LinearParam, SelfAttentionParam, RmsNormParam, ActivationParam, ElewiseParam, KvCacheParam, SoftmaxParam, and LayerNormParam |
libatb.so |
atb/train_op_params.h |
Definition of training operator parameters, including FastSoftMaxParam/FastSoftMaxGradParam, LaserAttentionParam/LaserAttentionGradParam, StridedBatchMatmulParam, and RopeGradParam |
libatb_train.so |
atb/common_op_params.h |
Definition of common operator parameters, including EventParam (stream synchronization event) and IfCondParam (runtime condition branch) |
libatb.so |
atb/graph_op_builder.h |
GraphOpBuilder API, which combines multiple operators into a computational graph |
libatb.so |
atb/utils.h |
Utils class, which provides static methods such as version query (GetAtbVersion), tensor size/element quantity calculation, quantization parameter conversion, and log level setting |
libatb.so |
atb/atb_acl.h |
aclnn operator APIs in C, used to call functions in two phases for specific heterogeneous operators (such as AtbMLA, AtbFusedAddTopkDiv, AtbRingMLA, and AtbPagedCacheLoad) |
libatb.so |
atb/comm.h |
Communicator APIs, which provide HCCL communicator management functions such as CreateHcclComm, CreateHcclCommByRankTableFile, and DestoryHcclComm |
libatb.so |