1. Mapping between CANN and driver versions
2. CANN package mapping
CANN 9.0.0 portfolio contains three packages: Toolkit, operator (ops), and acceleration (NNAL). The following table describes the mapping between the CANN portfolio and subpackages, and these six subpackages allow on-demand installation and independent upgrades:
[object Object][object Object]
[object Object]For details about how to upgrade a subpackage independently, see .
3. Mapping between CANN ops and Toolkit
The CANN ops package has been decoupled from Toolkit and can be upgraded independently. You can install the CANN ops package on demand.
CANN now supports Ascend 950PR (Atlas 350 accelerator cards) and has added the following key features: support for FP8/MxFP8/MxFP4, SIMD+SIMT hybrid programming for AscendC, and CCU communication acceleration for collective communication.
With the new apt and pip, installation modes now extend to four: conda, yum, apt, and pip. In addition, you can now download and install CANN in one stop, reducing the time required for obtaining and deploying software from 2 hours to 45 minutes. For details, see the .
On Atlas A2 and Atlas A3 products, DSA operators have further improved pipeline parallelism, enhancing SFAG operator performance by 2x–6x+ and boosting model efficiency by 1.5x+. The new deterministic computation capabilities deliver high result consistency. For details, see the .
The Atlas A2 and Atlas A3 products now support the Block-wise Sparse Attention (BSA) operator. Compared with generic Flash Attention, BSA reduces computation by 90% and speeds up execution 5x at 90% sparsity. For Wan2.2 and HunyuanVideo 1.5, the model performance is improved by 1.7x and 1.5x respectively when the sparsity rate is 70%. For details, visit .
HCCL supports the batch communication combination mechanism (HcclGroupStart and HcclGroupEnd APIs) on Atlas A2 and Atlas A3 products. This mechanism allows multiple communication operations to be combined and submitted together, improving the execution efficiency in small-message and high-frequency communication scenarios. For details, visit and .
- CANN improves the usability of installation and deployment.
- Added the apt-get and pip online installation methods.
- Added the combined package to CANN. This package consists of the Ascend NPU driver and Toolkit, and supports one-click installation and installation by specifying the
[object Object]parameter.
- Added the Ascend-cann-950-ops package to the CANN ops package to support Ascend 950PR operators.
- Adapted to different Ascend hardware products, and added OS compatibility.
- Atlas 350 accelerator cards are supported, and veLinux 2.0 and Alinux 3 are adapted.
- Atlas 200T A2 Box16 adapts to Ubuntu 22.04.5.
- Atlas 800T A2, Atlas 800I A2, and Atlas 900 A2 PoDc adapted to UOS V25 (kernel 6.6).
- Atlas 800I A2 adapted to AntOS (kernel 6.6).
- Atlas 800T A3 and Atlas 900 A3 SuperPoD adapted to CTyunOS4-25.07 and UOS V25 (kernel 6.6).
- Added support for NumPy 2.x for CANN, allowing CANN to run with later versions of NumPy.
- Open source LSTM operators:
- Low-bit operators and fused operators supported more data types, such as fp8/mxfp8/hifp8/mxfp4, and different quantization and combination modes, such as per-tensor/per-channel/per-token/per-group/per-block.
- Open source operators supported Ascend 950PR:
- Ascend 950PR supported the Parallel Welford and ultra-long axis binary accumulation algorithms to improve the numerical stability and computing precision of Norm operators. This feature involves the following operators:
- The following MM fusion operators are added to Ascend 950PR:
- The QuantBatchMatmul operator supports the mxfp8 weightNz feature based on Ascend 950PR, improving the network inference performance ().
- The following SIMD/SIMT homogeneous programming operators are supported:
- Added the operator development sample for <<<>>> ().
- Supported the generation of static libraries during compilation. The released independent NN component package contains the static library ().
- Added the capability of compiling specific operator kernels based on templates ().
- Provided best practices for programming MatMul, convolution, norm, and hash operators:
- LSTM/LSTMCell can be called through the aclnn APIs, which significantly reduces the compile time on the host when models that use LSTM operators, such as speech processing models, are called through PyTorch APIs.
- Optimized memory usage for index APIs such as aclnnScatter, aclnnScatterValue, and aclnnScatterAdd when dim!=-1. In model training or inference scenarios where the scatter or scatter_add API is used, there are obvious memory benefits when dim!=-1.
- The capabilities of transformer-related operators on the Atlas A3 products are improved as follows:
- DSA operators supported deterministic computation. Operators such as SparseFlashAttentionGrad, LightningIndexerGrad, and DenseLightningIndexerGradKLLoss supported deterministic computation. For DeepSeek V3.2 model training, deterministic computation is enabled, and the attention part can ensure that the results of multiple executions with the same input are consistent.
- Added MatmulAllToAll and AllToAllMatmul operators. For models trained using Ulysses parallelism with ultra-long sequences of M-level, these operators can improve the model performance.
- GmmSwigluQuantV2 supported the i4 input and i8 output and dynamic tiling optimization for A4W4. Gmm A4W4 supported dynamic tiling optimization for NZ transpose.
- When MoE permutation RoutingMaps support BF16 tokens, fp32 probs are input. The MoeInitRoutingQuantV2 operator supported INT4 and SmoothQuant.
- The M-RoPE fused operator generalization mrope_section supported [16, 16, 16, 16] and [24, 20, 20], and the prefill performance of the mrope operator was optimized.
- The aclnnScatterPaKvCache supported the FIA operator in the NZ scenario. The model side can use the FIA operator to obtain the performance benefits of the NZ scenario.
- The basic APIs of the FAG SAMEAB and BN2 templates were optimized and rewritten. Compared with CANN 8.5.0, the generalization test performance is improved by more than 1% and 10% on average.
- The deterministic performance of FAG in certain scenarios (layout=BSND/BNSD/SBH, sparsemode=0/2/3, dtype=bfloat16, S1=S2) was optimized.
- The forward and backward Floyd attention operators (Fuse Floyd Attention/Fuse Floyd Attention Grad) were added. Compared with the small operator combination in the Floyd structure, 10 matmul operators are arranged in pipeline mode, improving the fusion performance by more than 30% and reducing the memory usage by more than 70%.
- The RingAttentionUpdate operator supported Ascend 950PR.
- Image operators supported Ascend 950PR, covering ResizeNearestNeighborV2, ResizeNearestNeighborV2Grad, UpsampleNearest, and UpsampleNearest3d ( ).
- The CI enhanced the build and UT identification of Ascend 950PR (, and ).
- Operator samples can be executed on the simulator. Sample compilation and execution verification can be performed without relying on real hardware, facilitating Ascend 950PR adaptation for pre-integration and regression testing ( ).
- The RoiPoolingWithArgMax and RoiPoolingGradWithArgMax operators were added to support the Ascend 950PR and enable the faster-rcnn model.
- CIou can be called through the aclnn API, and the mmcv function can be enabled on the Ascend 950PR.
- The Col2im operator was added to support the Ascend 950PR and enable the training of the BEVfusion model for intelligent driving.
- The Rasterizer operator was added to implement rasterization calculation. It calculates the minimum depth of each pixel and the corresponding patch index based on the 3D spatial points and surfaces. This operator can be used to improve the model performance in multi-modal 3D generation scenarios.
- Mathematical operations, tensor transformation, and random number generation supported Ascend 950PR, covering Erfc, Sinh, Asin, Atanh, BitwiseXor, Asinh, Cosh, Scale, Tan, Acos, Acosh, and more ().
- APIs such as aclnnMul, aclnnMuls, aclnnAdd, aclnnAdds, and aclnnSum supported discontinuous inputs, improving the performance of related APIs and optimizing the memory usage ().
- Compared with Atlas A3, the Sort operator supported the UINT16, UINT32, and UINT64 types on Ascend 950PR. The average performance of data types supported by Atlas A3 is improved by more than 1.5 times on Ascend 950PR ( ).
- The performance of discrete tensor transformation operators, including Pad, Transpose, and AsStrided ( ), is optimized.
- Operator samples can be executed on the simulator. This allows sample build and execution verification without depending on real hardware, facilitating joint commissioning and regression ( ) of Ascend 950PR adaptation.
- The operator project supported parallel kernel compilation ().
- The operator project log was optimized ().
- The operator CI supports incremental UT and smoke tests ( ).
- The operator build script supported the export of precompiled files, which simplifies the locating of operator problems ().
- Docker-based deployment was supported ().
- The ChunkCat fusion operator was added. In the FSDP2 framework, this operator is required to merge weights during training. Compared with small operators, this operator reduces the memory usage by 30% and reduces the peak memory usage of the model, while improving the model performance.
- The aclnnDiv/aclnnDivs API supported mixed data types as inputs, optimizing the memory usage of the operator.
- The aclnn API for the logspace operator was added to improve the API usability.
- The aclnnReplicationPad3d API supported the bfloat16 data type, improving the API usability in multi-modal generation scenarios.
- The aclnnReplicationPad2dBackward API memory was optimized to reduce the peak memory usage and improve the operator usability.
- Communication operators supported Ascend 950PR, covering Allgather, AllgatherV, Allreduce, AlltoAll, AlltoAllV, Broadcast, Reduce, ReduceScatter, ReduceScatterV, Scatter and SendRecv ().
- With AI CPUs of Atlas A3 SuperPoDs, AlltoAll operators supported symmetric memory ().
- With AI CPUs of Atlas A3 series products, ReduceScatter&AllReduce operators supported batch consistency ().
- With ReduceScatter&AllReduce&AllGather&AlltoAll operators of Atlas A3 SuperPoDs, superkernel () between multiple machines was supported.
- Offline build was supported to improve the build friendliness ().
- On the Atlas A2 and Atlas A3 products, HCCL supported batch communication combination (HcclGroupStart and HcclGroupEnd APIs), which allows multiple communication operations to be combined and submitted for execution in a unified manner. This mechanism improves the execution efficiency in small-message and high-frequency communication scenarios and supports unified scheduling of multiple point-to-point communication operations.
- On the Atlas A2 products, AI CPU acceleration was supported for unilateral communication. In PD disaggregation scenarios, compared with the current host CPU scheduling mode, the performance of the AI CPU acceleration mode is improved by more than 50% when the size of the kvCache data pulled for inference is less than 128 KB.
- The performance of the cross-machine deterministic algorithm on the Atlas A2 products was optimized, improving the performance by 2x. Compared with the scenario where deterministic computation is not enabled, the deterioration is lower than 30%. In deterministic scenarios, there are obvious benefits.
- The Atlas A3 products supported the configuration of the HCCL expansion mode (such as AI CPU and AIV) by communicator, improving configuration flexibility.
- On the Atlas A3 products, the allreduce, allgather, reducescatter, and alltoall operators supported symmetric memory, improving the operator execution performance.
- On Atlas A3 products, the AllReduce and ReduceScatter operators supported batch consistency in AI CPU expansion mode.
- Supported deployment across multiple containers (without sharing the same NPU) on the same physical machine. During communicator initialization, a random port is selected. During multi-thread concurrent communication, contexts are isolated.
- Improved maintenance and test capabilities. For example, the output content of ErrorMsg was optimized, and time tracing was added to the AI CPU expansion.
- Unilateral communication: The Atlas A3 training products and Atlas A3 inference products supported the automatic link setup mode, and the connect API () does not need to be explicitly called.
- Unilateral communication: The Atlas A3 training products and Atlas A3 inference products supported the global unified addressing mode for the fabric memory, and supported the direct KV cache transmission capability of D2rH using HCCS, improving the transmission efficiency ().
- Core operators (such as Activation, MultiLatentAttention, and LayerNorm) supported Ascend 950PR.
- In the entire-graph offload mode, parameters of the plugin operator can be updated.
- The torch extension module was added to the SiP library. The C++ APIs can be bound to the Torch Library, and more than 40 operators, such as FFT, BLAS, and SOLVER, can be called using Python.
- The memory allocation and management mechanism was optimized. Resource pools facilitate efficient dynamic memory management. In FFT scenarios, handles are further reused, reducing resource waste and repeated resource application and release.
- ES graph construction provided samples ( and ) for multiple scenarios.
- ES graph construction Readable Dump supported user-friendly display of subgraphs.
- Enhanced the custom pass for image modification. Added the capability of modifying images after the original image was optimized.
- Supported memory reuse for the continuous memory without padding in the multi-stream scenario of graph mode.
- ES graph construction supported the generation of valid overloaded C++ APIs () based on the consumption of historical prototypes.
- Supported code optimization in the operator-level core control scenarios ().
- When the output node of a model is deleted during graph fusion, the framework can identify and update the model output ().
- Optimized common subexpression elimination ().
- Provided the dump switch to enable L0 and L1 exception dump ().
- Optimized the error message ().
- Provided the following end-to-end samples: dynamic profiles () (), integrating custom operators into graphs (), and custom pass for fusing Matmul and Add into GEMM based on the graph API ().
- Optimized the compilation project () ().
- Deterministic and strong consistency configurations were supported: ge.deterministicLevel.
- Supported TensorMove elimination.
- In online scenarios, users can use the graph-level option ge.outputDatatype to specify the model output data type.
- Removed the singleton and lock on the execution path to improve dispatch performance.
- Ascend 950PR supported the SIMD programming mode and provided more than to ensure cross-generation compatibility, implementing smooth operator migration between Atlas A2 and Atlas A3 products.
- Ascend 950PR introduced the Reg-based programming mode and provided for Reg data movement, basic arithmetic operations, reduction computation, and synchronization control.
- Atlas A2 products, Atlas A3 products, and Ascend 950PR supported , array-based memory allocation, and pointer-based computing APIs, providing native C programming experience.
- Ascend 950PR supported SIMD and SIMT hybrid programming and provided about 700 , including basic APIs such as warp, atomic, basic mathematical computation, and type conversion.
- Ascend 950PR supported the CCU communication interfaces for high-level communication APIs and provided based on the CCU. The high-level Matmul API supported , reducing memory usage by half and magnifying the compute throughput.
- About 260 new and compatible samples were added for Ascend 950PR, including SIMT and SIMD samples (framework, basic APIs, high-level APIs, and best practices). The was adjusted according to the programming model and sample type to improve the readability.
- Migrated the AscendC samples in the and the samples in the to the , and used the <<<>>> calling mode.
- Fusion compilation and <<<>>> calling supported the and .
- Worked with the BiSheng Compiler to optimize the fusion compilation performance.
- Supported the , including element-wise, TileScalar, fixed pipeline, memory access, complex operations, convolution instructions, and quantization instructions, as well as the of the corresponding instructions.
- Added SDMA() and URMA() asynchronous communication instructions, and multi-device P2P/P2MP/semaphore synchronization instructions ().
- Added the for performance simulation.
- Added the debugging command TPRINT to print tile data in Vec, Mat, and Acc storage and tensor data in GM. (, , )
- Added the performance tuning commands TPUSH/TPOP (, , ) and TPREFETCH (, ).
- The runtime supported Ascend 950PR.
- Enhanced usability
The stream specification in the AclGraph scenario can be expanded to 64K, resolving issues such as insufficient resources for foundation models. ()
The event specifications in the AclGraph scenario can be expanded. The expansion depends only on the device memory ().
Provided APIs for querying the package version number. The numeric version number and string version number are returned based on the package name. The APIs are as follows:
The priority of a specified stream can be queried. The API is as follows:
The flag set during stream creation can be queried. The API is as follows:
The universally unique identifier (UUID) of a device can be obtained. The API is as follows:
The start address and size of the memory block to which the address to be queried belongs can be obtained. The API is as follows:
The parameters for strong consistency computing can be set and queried. The APIs are as follows:
- Released the runtime programming guide ().
- Released the runtime Ascend 950PR documentation ().
- msProf supported the aicore-metrics option to collect custom PMU metrics. ()
- The HCCL performance test tool supported the FP64 data type ().
- The HCCL performance test tool supported the msfp8 data type ().
- Supported the HIF8/FP8/FP4/MXFP8/MXFP4 quantization data types and the HIF8 OFMR quantization algorithm ().
- Supported the HIF8 quantile quantization algorithm ().
The following APIs in the transformer library are marked as deprecated in CANN 9.0.0 and will be deleted in versions released after March 30, 2027.
- The aclnnGroupedMatMulAllReduce API is deprecated and replaced with aclnnMatmulAllReduce.
- The aclnnGroupedMatmul, aclnnGroupedMatmulV2, aclnnGroupedMatmulV3, and aclnnGroupedMatmulV4 APIs are deprecated and replaced with aclnnGroupedMatmulV5.
- The aclnnFusedInferAttentionScore/aclnnFusedInferAttentionScoreV2/aclnnFusedInferAttentionScoreV3 API is deprecated and replaced with aclnnFusedInferAttentionScoreV4.
- The aclnnIncreFlashAttention/aclnnIncreFlashAttentionV2/aclnnIncreFlashAttentionV3 API is deprecated and replaced with aclnnIncreFlashAttentionV4.
- The aclnnPromptFlashAttention/aclnnPromptFlashAttentionV2 API is deprecated and replaced with aclnnPromptFlashAttentionV3.
- The aclnnMlaProlog/aclnnMlaPrologV2WeightNz API is deprecated and replaced with aclnnMlaPrologV3WeightNz.
- The aclnnMatmulAllReduceAddRmsNorm API is deprecated and replaced with aclnnMatmulAllReduce and aclnnAddRmsNorm.
- The aclnnQuantMatmulAllReduceAddRmsNorm API is deprecated and replaced with aclnnQuantMatmulAllReduceV2 and aclnnAddRmsNorm.
- The aclnnWeightQuantMatmulAllReduceAddRmsNorm API is deprecated and replaced with aclnnWeightQuantMatmulAllReduce and aclnnAddRmsNorm.
- The aclnnInplaceQuantMatmulAllReduceAddRmsNorm API is deprecated and replaced with aclnnQuantMatmulAllReduceV2 and aclnnAddRmsNorm.
- The aclnnInplaceMatmulAllReduceAddRmsNorm API is deprecated and replaced with aclnnMatmulAllReduce and aclnnAddRmsNorm.
- The aclnnInplaceWeightQuantMatmulAllReduceAddRmsNorm API is deprecated and replaced with aclnnWeightQuantMatmulAllReduce and aclnnAddRmsNorm.
The following features of the model compression tool are deprecated. The deprecated features will be deleted in versions later than CANN 9.0.0.
- NUQ
- Mixed precision
- Calibrated approximation
- int4 QAT
- All features of amct_mindspore
Issue 1: For a model that uses the int64 operator in the communicator, when a link disconnection occurs, the time required for fast recovery increases from minutes to more than 10 minutes, affecting the MTTR. [Involved Version] CANN 9.0.0 [Impact] The int64 operator does not support the re-execution process and cannot be restored using the step-based fast recovery method. As a result, tasks cannot be re-executed under disconnections. [Workaround] Do not use the collective communication operator of int64 type. Instead, use operators of other types as temporary replacements.
Issue 2: During the recovery of a training job, the sendrecv operator is suspended. As a result, the process-level recovery fails, affecting the MTTR. [Involved Version] CANN 8.5.0 [Impact] In cross-SuperPoD scenarios, the sendrecv operator is occasionally suspended, and the fast recovery fails. [Workaround] In logical cross-SuperPoD scenarios, you can use the HCCS link to avoid this problem. Currently, no workaround is available for the physical cross-SuperPoD scenarios.
- Resolved the issue that the error message "ld.lld: error: undefined symbol: CheckLogLevel" is displayed during the build of CATLASS operators.
- Resolved the issue that the error code "EZ9999" is reported during the execution of the PadV3Grad operator on the Atlas 900 A2 hardware because the operator does not support the padding input of [16,16,16,16] format.
- Resolved the issue where the MatMul operator produced NaN results during matrix multiplication (32162, 4096)@(32162,32768).
- Resolved the issue that the FA operator consistently produced NaN values in the MIMO-V2 model during training under the MindSpeed-LLM framework.
- Resolved the issue that the aclnnQuantMatmul operator reported an error indicating that the shapes are inconsistent when the input is a slice parameter.
- Resolved the issue where the FAG operator caused model calling timeouts due to losing tail data in data tiling in the (1,32,641,128) no mask scenario.
- Resolved the issue where a cluster task failed because the HCCL communication was suspended when the reduce communication operator was used in an environment where two driver versions coexisted.
Changed the CANN Quick Start to the "Environment Setup" section and renamed it CANN Quick Installation.
- Added the introduction to SIMD and SIMT, and added the section for getting started with SIMT programming.
- Added the sections "AI Core SIMT Programming" and "Hybrid Programming of SIMD and SIMT" to the programming model in the programming guide.
- Added the section "AI Core SIMT Compilation" to the compilation and running section in the programming guide.
- Added the SIMT built-in keywords and C APIs at the SIMT language extension layer in the programming guide.
- Added the sections "SIMT Operator Implementation" and "SIMD and SIMT Hybrid Operator Implementation" to Operator Practice References.
- Added the section "SIMD and SIMT Operator Performance Optimization" to Operator Practice References.
- Added the section "SIMT APIs" to API Reference.
- Added the compatibility migration guide (migration from the 220x architecture to the 351x architecture).
- Added 90+ Reg programming APIs. Reg vector computation APIs are designed for the Regbase architecture. You can use these APIs to directly operate registers involved in vector computation on the chips, achieving greater flexibility and better performance.
- The Ascend community added a video zone for Ascend C operator development, specifically, the execution process of Cube and Vector operators.
- Communication Operator Development was extracted from Huawei Collective Communication Library (HCCL) and released as an independent document under the programming guide.
- Added the section "AIV Communication Operator Development."
Adjusted the level-1 directory by function and feature:
- Promoted the "Runtime Management" section to one level higher.
- Added the development guide for more runtime features, such as AscendCL graphs, multi-device programming, and inter-process communication.
- Moved the API call process description to the sections of specific functions and features.
- Distinguished built-in and custom operators in each single-operator calling section.
- Moved the media data processing and feature vector search functions to the "Dedicated Accelerator" section, and added the description of the random number generation function to this section.
- Split AscendCL API Reference by component into runtime APIs, GE APIs, DVPP APIs, and feature vector search APIs.
- Adjusted the outline: Integrated and restructured the content, and moved the development-state sections to the programming guide.
- Incorporated AutoFuse User Guide into the "AutoFuse" section of Graph Development.
- Added the section "Appendix > Session to GeSession Migration Guide" to Graph Development.
- Added the section "Programming Guide > Integrating a Triton Operator into a Graph".
- Classified APIs by language (C++, Python, and C). Python APIs are new to this documentation.
- Added "Model Management and Single-Operator Calling APIs" to the C APIs. The content comes from the original acl APIs.
- Added the section "Configuring Resource Information Through Rank Table (Atlas 350 Accelerator Cards)" to the section "Reference > Cluster Information Configuration".
- Added the section "AIV Operator Development" to the section "Communication Operator Development".
Changed Memory Leak Detection Tool (msLeaks) to Memory Analysis Tools.
For details about the fixed vulnerabilities in open-source and third-party software, see .
You can now upgrade subpackages on demand.
For example, if you have installed the Ascend-CANN-Toolkit and Ascend-CANN-ops packages of version a.b.c, and the communication library needs to be updated to version x.y.z, you can upgrade the cann-hixl subpackage independently. Replace a.b.c and x.y.z with the actual version numbers.
Before upgrading a subpackage, ensure that the Ascend-CANN-Toolkit and Ascend-CANN-ops packages of the matching version have been installed. If the packages are not installed, install them by referring to .
Subpackage upgrade paths must be the same as that of the main package. The upgrade operations of all subpackages are the same. The following uses [object Object] as an example.
Upgrade scenario 1:
The Ascend-CANN-Toolkit and Ascend-CANN-ops packages are installed in the default path (that is, not using [object Object] for installation).
Upgrade scenario 2:
The Ascend-CANN-Toolkit and Ascend-CANN-ops packages are installed in the specified path [object Object] (that is, specifying [object Object] during installation).