Release Notes
1. Version Overview
MindStudio is a full-process development toolchain built by Ascend for AI developers, covering core development stages such as operator development, accuracy debugging, performance tuning, and visual analysis. This version R&D pain points including model training accuracy issues, performance bottlenecks, quantization precision tuning difficulties, and Ascend 950 operator debugging and tuning. It continuously simplifies the Ascend AI development process, lowers the development barrier, improves quantization efficiency, enhances operator tuning and troubleshooting efficiency, and empowers developers to rapidly deploy AI apps, driving large-scale innovation in the Ascend ecosystem.
Training development tools: To address pain points such as multi-core scheduling imbalance in model training, training-inference inconsistency in reinforcement learning, difficulty in detecting anomalies during long-term training, prominent host bottlenecks, and high costs of troubleshooting complex issues, the new MindStudio version upgrades the training toolchain. Focusing on three scenarios—reinforcement learning, Agent intelligent analysis, and HostBound problem localization—it enables efficient end-to-end localization of training accuracy, performance, and memory issues, lowers the tuning barrier, and improves the stability and iteration efficiency of model training.
Inference development tools: To address development pain points such as long deployment cycles for model inference, high manual tuning costs, difficulty in parameter selection, and a steep learning curve, the new MindStudio version optimizes four key capabilities—accuracy data collection, quantization, simulation, and modeling—simplifying the entire inference development process, lowering the deployment barrier, and improving quantization tuning efficiency.
Operator development tools: To address the daily development pain points faced by Ascend 950 developers—such as a high barrier to debugging operators on the new architecture, incomplete anomaly detection capabilities, and time-consuming hardware performance tuning analysis—the new MindStudio version refines the full-process operator toolchain adapted for the Ascend 950. It delivers capability upgrades across three dimensions: anomaly detection, issue debugging, and performance tuning, thereby reducing the cost of operator development and shortening the iteration cycle for Ascend 950 operators.
2. Version Compatibility
| MindStudio | CANN | TorchNPU | Ascend HDK |
|---|---|---|---|
| 26.1.x | 9.1.0 | 26.1.0 | 26.1.0 |
3. Version Compatibility Notes
"Y" indicates supported, and "/" indicates not supported.
| MindStudio | CANN | ||
|---|---|---|---|
| 8.5.0 | 9.0.0 | 9.1.0 | |
| 8.3.x | Y | / | / |
| 26.0.x | Y | Y | / |
| 26.1.x | Y | Y | Y |
| MindStudio | TorchNPU | ||
|---|---|---|---|
| 2.3.0 | 26.0.0 | 26.1.0 | |
| 8.3.x | Y | / | / |
| 26.0.x | Y | Y | / |
| 26.1.x | Y | Y | Y |
| MindStudio | Ascend HDK | ||
|---|---|---|---|
| 25.5.x | 26.0.RC1/25.7.RC1 | 26.1.0 | |
| 8.3.x | Y | / | / |
| 26.0.x | Y | Y | / |
| 26.1.x | Y | Y | Y |
4. New Features
| Tool | Feature | Description |
|---|---|---|
| msKPP | compile.json generation | Added support for generating compile.json during build to record compile options and dependencies, enabling integration with performance analysis and CI toolchains. |
| msKPP | Unified build adaptation | Unified the build entry script to work with the new unified build image; UT environments are synchronized accordingly, improving build stability and CI compatibility. |
| msKPP | clang-tidy integration | Introduced clang-tidy configuration and a pre-commit hook to enforce static analysis and clean code standards before each commit. |
| msKPP | Unit test expansion | Added new unit tests for the core msKPP modules, increasing test coverage and strengthening regression protection. |
| msKPP | CI version tagging | Enhanced CI pipelines to support release version numbering based on tags, ensuring package versions align with source code versions. |
| msOpGen | Pip installation optimization | Replaced the legacy pip install approach with an entry_points-based installation to resolve residual file issues after pip uninstall. The msopst.ini configuration file is relocated to the standard configuration directory. Custom scripts relying on the old paths must be updated accordingly. |
| msOpGen | Unified build promotion | Elevated build.py as the formal unified build entry, with robust submodule checkout to pinned versions, improving first-time build success rates and usability. Also adapts to the new unified build image. |
| msOpGen | Loosened security checks | Removed symlink validation, file ownership checks, and path length restrictions during installation, lowering deployment barriers in constrained environments. |
| msOpGen | Quick start updates | Adapted to compilation interface changes in CANN 9.0.0; updates build commands and examples in quick start to ensure generated operator projects can be compiled properly under the latest CANN. |
| msOpGen | MindStudio branding & pre-commit | Added support for MindStudio startup logo and version info display in the CLI tool; added a pre-commit configuration file for automated pre-commit checks. |
| msOpGen | CI version alignment | Aligned CI packaging with tag- or branch-based versioning to guarantee that the released package version matches the actual code version. |
| msDebug | Ascend 950PR/DT on-board debugging | Added support for on-board debugging of AscendC operators on Ascend 950PR/DT chips, including hardware/software breakpoint management, memory/variable/register display, step-over/in, interrupt execution, multicore and thread switching, and debug info display. Supports breakpoints in simd_vf scenarios with automatic selection of software or hardware breakpoints based on the breakpoint location. |
| msDebug | Ascend 950PR/DT coredump analysis | Added support for coredump analysis on Ascend 950PR/DT, including call stack unwinding, error register display, and thread info display/switching. Automatically retrieves kernel objects from core files to assist symbol resolution. |
| msDebug | Host GM memory read | Added support for direct reading of device global memory without switching to the device when a breakpoint is hit on the host, facilitating rapid issue diagnosis in mixed host/device debugging scenarios. |
| msDebug | Enhanced thread information display | Added detailed thread info and thread-switching support, with core-file thread display. The MI stopped event now outputs the current core’s thread-dim info to clarify the debug context. |
| msDebug | Optimized register display | Registers are now categorized and displayed by type (AIC, AIV, SIMT, SIMD) with a unified print order. Added read support for S/R general-purpose registers, merged core-file parsing with on-board register read logic, and supplemented error-register display for next-generation chips. |
| msDebug | Automatic breakpoint type identification | The MI interface now provides a flag indicating whether a breakpoint lies in a SIMT region. The debugger can automatically distinguish soft vs. hard breakpoints based on this info, improving success rate and reducing interruptions due to type mismatches. |
| msDebug | Usability and build improvements | Unified the build entry script, supporting CMake 4.x and the new unified build image, and removing the CMake upper-version limit for compatibility with the latest build environments. Added MindStudio LOGO and version info to the CLI tool, adapted to the new Asc function registration method, and added support for aclrt operator without rigid dependency on the runtime interface. |
| msSanitizer | DCCI missing detection | Added detection for missing Data Cache Consistency Instructions (DCCI) via the --check-dcci=yes command-line option to automatically identify operators missing DCCI instructions, helping users catch potential data errors caused by cache incoherency early. When DCCI detection is enabled, instruction filtering and race detection are automatically turned off to avoid interference. |
| msSanitizer | Cross-NPU race detection | Introduced a complete cross-NPU race detection flow, enabled via --check-cross-npu-races=yes. It detects data races in shared-memory scenarios, supporting msTX soft-sync semantics (barrier/signal), and covering data-race analysis in inter-card communication. |
| msSanitizer | SIMT architecture detection enhancements | Extended SIMT detection with multiple capabilities: uninitialization detection now supports SIMT architecture and improves accuracy via instruction replay; SIMT race detection supports competitive analysis across multiple simt_vf functions, identifies UB races between simt and main_scalar, and detects unmatched sync_threads pairs; simt_call/simt_start/simt_end is now supported with enhanced call stack tracing for easier issue localization. |
| msSanitizer | Read/Write access control | Added read/write permission control to both SIMD out-of-bounds and SIMT detection. Users can specify via --check-level to detect only read, only write, or both directions, improving flexibility and reducing false positives. |
| msSanitizer | GM buffer safe zone out-of-bounds write detection | Added a protected safe zone at the end of Global Memory buffers to effectively detect write accesses crossing buffer boundaries, covering typical out-of-bounds scenarios like Tiling-offload AICPU. Safe zone size is configurable via --padding. |
| msSanitizer | Synchronization detection enhancements | Added operator hang detection to identify execution stalls caused by missing synchronization primitives under sync detection mode; supplemented race detection algorithms for get_buf/rls_buf synchronization semantics across pipelines and cores, reducing missed races. |
| msSanitizer | Runtime Ctrl-c interrupt | Added support for Ctrl-c signal interruption during operator execution, allowing users to gracefully exit when detection runs too long or needs early termination, improving interactive experience. |
| msSanitizer | Usability and build optimizations | Tool logging is now disabled by default for better performance (Users can re-enable via --log-level as needed); optimized help information with typical usage examples; unified build entry scripts with compatibility for CMake 4.x and new unified build images; CLI tool now displays product LOGO and version info. |
| msOpProf | Ascend 950PR/DT on-board instruction-level pipeline timeline | Added support for collecting and generating instruction-level pipeline timelines on Ascend 950PR/DT chips, visualizing execution timing of each instruction across pipeline stages to help developers quickly identify pipeline bottlenecks and bubbles. |
| msOpProf | SIMT warp-level pipeline chart and stall analysis | Added warp-level pipeline visualization for SIMT operators, showing time distribution across thread warps. Included automatic Top Stall Reason attribution to highlight primary stalls and hot spots. |
| msOpProf | Enhanced memory heatmap metrics | Refined instruction-level transfer paths via dynamic instrumentation, supplementing multi-directional memory bandwidth data (UB→GM, Dcache→GM, UB→L1, L1→UB). Expanded heatmap coverage for comprehensive memory bottleneck analysis. |
| msOpProf | Scalar overhead in simulation pipeline | Introduced scalar instruction overhead display (e.g., decode, issue) in simulation pipeline charts, distinguishing it from compute time to better identify scalar-side bottlenecks. Supported on both A2 and A3 chips. |
| msOpProf | Refined dynamic instrumentation & SIMT stubs | Added dynamic stubs for SIMT instructions, improving granularity of transfer path data collection and enhancing accuracy for on-board instruction-level pipeline charts and memory heatmaps. |
| msOpProf | AscendC API profiling markers | Added support for user-defined markers via AscendC API on select models, allowing developers to tag critical code sections. Marker data appears in pipeline charts for targeted performance analysis. |
| msOpProf | Usability improvements for custom markers | Enhanced display and UX of custom marker pipeline charts; warp pipeline charts now show active thread count to clarify parallelism and warp occupancy. |
| msOpProf | Build & usability optimizations | Unified build entry scripts, adapted to CMake 4.x; improved help info with typical use-case examples; added MindStudio LOGO and version info to CLI; included clang-tidy configuration for code checks. |
| msKL | Unified build scheme adaptation | Consolidated build entry scripts to support new unified build images, improving stability and ease of first-time builds. |
| msKL | pre-commit code checks | Added pre-commit configuration for automatic style and format validation prior to commit. |
| msKL | CI versioning adaptation | CI pipeline now supports version tag or branch-based release package versioning, ensuring package version matches actual codebase version. |
| msKL | Unit test supplement | Added unit tests for driver (test_driver.py) and config (test_config.py) modules to improve test coverage and enhance regression protection for core components. |
| msTX | Memory attribute specification API | Introduced msTX memory attribute API, allowing operators to explicitly mark memory region access attributes (read-only/write-only/read-write). This helps tools like msSanitizer accurately distinguish legal vs. abnormal accesses, significantly reducing false positives in memory error detection. |
| msTX | Torch NPU memory access report macro | Added TORCH_NPU_REPORT_MEM_ACCESS macro to report memory access information inside PyTorch operators. When used with Ascend PyTorch Profiler in mstx=True collection mode, it enables more fine-grained memory behavior analysis. |
| msTX | compile.json generation | Added support for generating compile.json during build to record compile options and dependencies, enabling integration with performance analysis and CI toolchains. |
| msTX | Unified build adaptation | Unified the build entry script to work with the new unified build image; UT environments are synchronized accordingly, improving build stability and CI compatibility. |
| msTX | clang-tidy | Added the clang-tidy configuration file and pre-commit hook to enable static analysis checks before code commits, enhancing overall code quality. |
| msTX | CI versioning with tags | Adapted CI build pipeline to support specifying release package version numbers based on tags, ensuring package version consistency with the source code. |
| msProf | Ascend 950PR/DT chip support: DPU scheduling & hardware sampling parsing | Added C-language parsing support for DPU task track and HCCL track data, enabling DPU operator dispatch/execution statistics; introduced DPU data loading processor flow and assembler export classes, supporting both DB format and timeline presentation; added support for the --reports parameter configuration for sampling items including CCU, BIU Perf, UB, and block detail on Ascend 950PR/DT chips. |
| msProf | Enhanced DB export capability | Extended msprof_db export to support new chip data tables such as CCU, SIO, UB, DPU, and AIcoreFreq, aligned with timeline data; refactored aicoreFreq data processing logic based on the lowPowerData structure. |
| msProf | Fusion task parsing & visualization | Adapted for Ascend 950PR/DT chips by adding Fusion Task parser, model, and C++ Host Parser/Calculator; added support for reading data from fusion_task.db and generating Chrome Tracing timeline units, including parameters like Task Id, Acc Id, Task Type, and CCU Die Id. |
| msProf | Host CPU NUMA metric collection & analysis | Added HostPlatformAnalysis class to process platform data collector output, supporting NUMA node and cross-node traffic statistics; optimized NUMA collector script with precise stop logic; improved user core-binding efficiency in Host Bound scenarios. |
| msProf | Compute-Communication overlap analysis & communication operator classification | Added support for algorithm bandwidth calculation and visualization; added the rankSize field to HCCL data tables; added overlap analysis in DB parsing; added support for 310P TS software scheduling task statistics and timeline rendering. |
| msProf | launchKernel interface profiling info parsing | Added support for parsing of additional profiling info reported via launchKernel interface, including SIMT operator gridDim/blockDim and scheduling mode, enhancing SIMT operator presentation in timeline. |
| msProf | Memory data V2 adaptation | Adapted to new hardware timestamp formats for HBM/LLC/MEM by moving time calculation logic to pre-DB storage stage; introduced versioned_struct_parser to uniformly handle binary parsing across different collection versions, ensuring data format compatibility. |
| msProf | Fault pattern library & tuning cases | Added a serving tuning fault pattern library covering typical scenarios: excessive scheduler latency, PrefixCache misses, KV Cache transfer impacting performance, SLO degradation, small-batch optimization, operator compilation overhead, and frequent synchronous API calls; provided large-model tuning examples. |
| msProbe | End-to-End NaN detection | Added support for end-to-end NaN detection based on hardware register overflow status, accurately identifying which specific operator in the entire network produces NaN. |
| msProbe | Service inference anomaly awareness | Added support for automatically detecting duplication/garbled-text issues during serving inference, providing real-time awareness of accuracy anomalies in inference systems. |
| msProbe | Custom operator collection | Added support for batch auto-registration of custom operators via YAML configuration files in msProbe, enabling collection of input/output information for user-defined operators throughout the network. |
| msProbe | verl hyperparameter comparison & validation | Added support for verl hyperparameter comparison and validation of critical parameters. |
| msProbe | Dirty data filtering in comparison/precheck | The comparison and precheck features now filter out dirty data, ensuring more accurate diff results. |
| msProbe | Device MD5 data collection | Added support for collecting device MD5 data. |
| msProbe | Quantization operator collection | Added support for collecting quantization operators under torch_npu. |
| msMonitor | Rank-targeted collection for nputrace in clusters | nputrace now supports the --rank-list parameter, allowing collection to be targeted to specified ranks in cluster scenarios. This avoids the overhead of full-device collection and enables precise profile data capture in multi-rank training/inference frameworks like VeRL and VLLM. |
| msMonitor | JSONL rotation via command-line | Added --json-rotate-log-files and --json-rotate-log-lines CLI parameters to control the number of rollover files and max lines per file when exporting JSONL-formatted data. These CLI options take precedence over environment variables, enabling flexible volume control without modifying the runtime environment. |
| msMonitor | Training/Inference use cases & FAQs | Added practical usage examples and FAQs for msMonitor in training and inference scenarios to address user feedback on lack of scenario-specific guidance, lowering the adoption barrier. |
| msPTI | Python Communication API exposure | The mspti-python package introduces CommunicationMonitor to replace the legacy HcclMonitor. Due to changes in how upper frameworks invoke communication operators, the original HcclMonitor could no longer collect data via hook interception. The new API restores and enhances communication operator collection, with UT/ST, samples, and documentation updated accordingly. |
| msPTI | msptiActivityIsEnabled API | Added msptiActivityIsEnabled to query whether a given Activity Kind is enabled at runtime, allowing tools to dynamically determine the collection configuration state. |
| msPTI | Data flush mechanism for Ascend 950PR/DT | Adapted to the updated data reporting logic from Ascend 950PR/DT driver to profiler components. Introduced the halProfDataFlush mechanism when closing Kernel, Communication, and other data collection types to prevent buffer data loss, ensuring complete collection results on Ascend 950PR/DT scenarios. |
| msMemScope | Host locked-page memory info collection | Added support for collecting allocation and release records of host locked-page memory requested by offload and other features, including memory size, timestamps, and call stacks, to assist in host memory analysis. |
| msMemScope | Cross-stream operator contention detection & custom operator instrumentation | Added detection capabilities for memory corruption issues caused by inter-operator contention in eager mode. Native PyTorch operators are supported by default; other operators can be manually marked to join the detection scope. |
| msMemScope | Scenario-based documentation & user guide enhancement | Structurally optimized the main README documentation, adding usage examples for features such as memory leak detection, memory comparison, memory block monitoring, memory breakdown, and inefficient memory identification. |
| msServiceProfiler | LLM inference online monitoring platform | Added ms-service-metric online monitoring capability. It collects vLLM-Ascend runtime metrics via lightweight hooks and exposes them in Prometheus format through the /metrics endpoint, which can be scraped by Prometheus and visualized with Grafana. This release covers EPLB hotspot aggregation/imbalance and expert movement latency, abnormal state counts (running→waiting rollback, KVCache block allocation failures, RPC exceptions), fine-grained stage latencies (forward/kernel_launch/non_forward), static and runtime memory monitoring, and multi-dimensional label injection (dp/role/phase) to enable independent observation of prefill/decode, different DP groups, and different roles. |
| msServiceProfiler | Ternary parameter derivation | Added two derived field types: ternary_factories and ternary_times to the auto-tuning tool, supporting derivation of a third parameter from two source fields (e.g., dp = world_size / (tp * pp)). Added two priority-aware constraint repair strategies (fixed and balanced) to automatically correct inconsistent parameter combinations during PSO search and rewrite particle positions, improving search space coverage and result fairness. |
| msServiceProfiler | Environment variable tuning & default throughput saturation for out-of-the-box models | Added environment variable dimension support in auto-tuning parameter search, and set throughput parameters to saturated values by default for out-of-the-box models, improving first-run tuning effectiveness and ease of use. |
| msServiceProfiler | Duplicate particle skipping | Introduced duplicate particle detection and skipping mechanism in the PSO search flow to avoid redundant evaluation of identical parameter combinations, saving tuning time. |
| msServiceProfiler | Monitoring dashboard smoke testing framework | Established a smoke testing framework for the ms-service-metric monitoring dashboard with comprehensive basic validation cases, covering fundamental field checks in basic and EPLB scenarios, along with fallback detection for hook failures at warning level and above. |
| msServiceProfiler | -c/--config command-line configuration parameter | Added the -c/--config command-line parameter to the auto-tuning tool, allowing explicit specification of a TOML configuration file from any path at startup. This file takes the highest priority, facilitating rapid configuration switching across different model and cluster scenarios. |
| msInsight | Enhanced active analysis for ftrace data in MindStudio Insight | Added support for ftrace collection in tracefs/debugfs mode, and ftrace DB output as well as joint import of profiling and ftrace data. In the Insight frontend, ftrace data is uniformly displayed as an Ftrace Task Summary, offering filtering and sorting capabilities by running CPU, process/thread, context switches, soft/hard interrupt counts and durations, and runnable/running/sleeping durations. It can also generate Excel analysis reports containing multiple worksheets and charts. |
| msInsight | CPU-filtered conversion in trace_convert | Added CPU range filtering capability to trace_convert, allowing conversion of only CPU Scheduling, IRQ/SoftIRQ, and Process Scheduling events for specified CPUs, reducing conversion time and output size for large-scale trace data. |
| msInsight | Joint import of profiling and ftrace Data | Added support for combined import of various data combinations including profiling text, profiling DB, ftrace text, and ftrace DB. After import, the System View displays corresponding analysis tabs based on data type, facilitating joint analysis of device-side profiling and host-side scheduling data. |
| msInsight | Potential leak tensor analysis in memsnapshot | Added a "Potential Leak Tensor" filter to the memory block table in memsnapshot, identifying memory blocks that have been allocated but not freed within the selected event interval, and displaying total, maximum, and minimum sizes of potential leak tensors. |
| msInsight | Memory pool segment detail display | The memory pool state diagram now supports independent segment-level detail views, including basic stats such as segment size, allocated size, gap size, block count, gap count, and max gap size. When allocation/map event context is available, it is displayed alongside; even when events are missing, segment-level stats remain accessible. |
| msInsight | Enhanced memory snapshot observability | Added reservedSize line chart support for memsnapshot, a loading progress indicator for snapshot parsing, and data-type differentiation capabilities for Snapshot-based comparison analysis, improving the overall data analysis experience for large-scale memory snapshots. |
| msInsight | Containerized deployment & Web access | Added MindStudio Insight Dockerfile and related files to enable Docker image building; provided streamer container start/stop scripts supporting HTTP, HTTPS+mTLS, data directory mounting, certificate directory mounting, fixed/dynamic port configurations, and other scenarios, enabling rapid Web analysis service setup in local or PoC environments. |
| msInsight | IPv6 access support | Frontend, backend, and foundation layers now support IPv6 address access, with launch parameters accepting valid IPv6 addresses. The JupyterLab plugin is also adapted for IPv6 scenarios, improving usability in pure IPv6 or dual-stack network environments. |
| msInsight | Enhanced timeline interaction | Timeline now supports hierarchical merging of multi-hardware metric units, adds a floating toolbar, and introduces features such as same-name unit pin-to-top, right-click time alignment, one-click communication operator alignment, secondary filtering in the search panel, and scrollbar and right-click menu, significantly improving issue diagnosis efficiency for multi-card and large-scale Timeline data. |
| msInsight | Optimized python call stack display | Python call stacks now support independent unit display, with performance optimizations for Python stack rendering to reduce visualization and interaction overhead for large-scale datasets. |
| msInsight | Enhanced network & communication data display | Added support for displaying network-related data from NIC and ROCE tables with unified unit presentation (Byte, Packet, etc.). The COMMUNICATION_OP table now adapts the deviceId column, enhancing query capabilities for communication units, search, event views, and discovery lists in multi-device scenarios. |
| msInsight | Operator and link analysis enhancement | System view dashboard: added per-trace detail view and operator detail view, allowing drill-down from aggregated statistics to specific API/operator invocation instances. Operator tuning: added Top Wall Reason and other analysis views, supporting stall top reason chart comparison. Kernel E2E: added backend tables and overall latency analysis capabilities to assist in locating time-consuming issues in operator dispatch and execution chains. |
| msInsight | Cluster analysis log persistence | Standard output and standard error during cluster analysis tool execution can be redirected to log files in the data folder, facilitating issue diagnosis when MindStudio Insight invokes cluster analysis capabilities. |
| msInsight | Scenario-based documentation and user guide enhancement | Supplemented operator tuning pipeline flowchart common issue guidance, verl scenario Snapshot data collection and analysis cases, PyTorch Snapshot analysis cases, ftrace tool user guide, containerized deployment documentation, system tuning data collection instructions, single-card dispatch analysis cases, and slow rank tuning cases, improving overall documentation path completeness and scenario-based enablement. |
| msprof-analyze | Agent mode adaptation | Adapted to agent invocation scenarios: --help now uniformly displays help information for all subcommands; --quiet suppresses progress bars and logs while outputting streamlined JSON; certain analysis capabilities return structured JSON for msagent to read DB for further analysis. Advisor logs are uniformly integrated into the common module, retaining only essential output in Agent mode. |
| msprof-analyze | Cluster analysis dependent solely on single-rank analysis.db | Adapted -m all/communication_matrix/communication_time to scenarios containing only analysis.db and profiler_info.json. When ranks are missing, skip according to required_db_keys and issue unified warnings, reducing data dependencies and transfer costs in ultra-large-scale cluster scenarios. |
| msprof-analyze | Operator and end-to-end MFU analysis | Added operator_mfurecipe, outputting kernel-level OperatorMFU details and module-level ModuleMFU aggregated statistics. Refactored MFUCalculator to consume operator FLOPs collected via with_flops=True + MSTX recording, calculating MFU based on kernel duration, input data types, and chip theoretical peak performance. |
| msprof-analyze | Cluster analysis summary export | Added the export_summary mode to cluster analysis for exporting per-device API statistics and kernel details as table deliverables, generating api_statistic.csv and kernel_details.csv. |
| msprof-analyze | Slow link analysis capability | Introduced slow link analysis to help identify communication link-level bottlenecks within the cluster. |
| msprof-analyze | Cluster analysis performance optimization | Optimized CommMatrixSum by pre-building the rank_set mapping, replacing row-wise lookups with DataFrame merge, using a single concat for mapper results, replacing iterrows() with itertuples(), applying max_depth=5 depth limit for DB scanning. Refactored HostInfoAnalysis into four stages (task building, concurrent scanning, single-rank scanning, and result merging) with ThreadPoolExecutor for concurrent multi-rank DB reads. Removed obsolete and unused data-reduction corruption code. |
| msModelSlim | Expanded model quantization coverage | Added quantization support for multiple text and MoE model series including DeepSeek-V4/Pro, Kimi K2.5/K2.6, MiniMax M2.7, GLM5.1/GLM4.7-Flash, InternVL3.5/MoE, and Step3.5-Flash. |
| msModelSlim | Multimodal & generative model quantization extension | Added quantization support for multimodal understanding models such as Qwen3-VL-MoE, Qwen-Image-Edit, and InternVL3.5, as well as generative models including Wan2.2 (video generation) and LongCat-Flash (image generation). Refactored the multimodal generative model quantization service to support scenario-specific optimizations like Wan2.2/HunyuanVideo expert sub-network splitting, covering both multimodal and AIGC scenarios. |
| msModelSlim | DualScale quantization algorithm | Introduced the DualScale quantization algorithm, which achieves improved quantization results on sensitive layers. |
| msModelSlim | Binary fallback auto-tuning strategy | Added a binary fallback auto-tuning strategy with an M-to-N automatic retry mechanism, improving auto-tuning efficiency and success rate. |
| msModelSlim | FA3 capability enhancements | Extended FA3 to support multi-rank quantization, mixed quantization configurations, MXFP4 per-block dynamic activation quantization, and MXFP8 format export, significantly expanding applicable scenarios for FA3. |
| msModelSlim | Multi-rank distributed quantization architecture enhancement | Added expert parallel mode support for multi-rank quantization. Integrated outlier suppression algorithms (FlexSmooth, FlexAWQSSZ) and AdaptRotation into DistributedTaskScheduler for multi-rank distributed execution. Added distributed saver queue load balancing and DFX observability capabilities. |
| msModelSlim | MXFP4 Quantization capability enhancements | Added support for four-over-six algorithm and ceil_x quantization method for MXFP4, along with mxfp8 mse_round optimizations to extend low-bit quantization accuracy. Added pack_fp4_to_uint8 weight packaging capability for streamlined weight encapsulation and transmission. |
| msModelSlim | Sensitive layer analysis enhancements | Added support for MSE Model Wise and MSE Layer Wise metrics for sensitive layer analysis. Added support for layer-level rollback in the Standing High algorithm to improve analysis accuracy and tuning flexibility. |
| msModelSlim | Quantization format & toolchain enhancements | Added compressed_tensors quantization format and weight conversion (int4↔bf16). Introduced YAML validation and dependency precheck for quantization configs. Added fast_ops_grapher tool for model structure extraction. Added mean aggregation in the minmax algorithm. |
| msModelSlim | Engineering system optimization | Unified the build entry script under the new build scheme for improved usability. Added support for pip install -e installation. Streamlined global security policies and refined documentation directory structure. |
| msModeling | Qwen3.5 Dense/MoE model adaptation | Added text input simulation support for Qwen3.5 Dense and Qwen3.5 MoE models, expanding coverage for key Qwen-series models and aligning with Q2 new-model adaptation goals. |
| msModeling | GLM-4 MoE model adaptation | Added simulation support for GLM-4 MoE, complementing MoE inference performance evaluation capabilities for the GLM series and improving integration efficiency for key models. |
| msModeling | DeepSeek-V4 model adaptation | Added DeepSeek-V4 model support in msModeling, covering end-to-end simulation for Flash/Pro editions. Added support for sparse/compressed attention, KV cache compression, and MTP inference features, reinforcing performance evaluation coverage for key DeepSeek-series models. |
| msModeling | GLM-5.2 IndexShare inference feature adaptation | Added GLM5-specific IndexShare auxiliary logic, supporting full-layer indexer execution and reuse of previous full-layer top-k indices by the shared layer. Extended MTPindexer_types support. |
| msModeling | GLM-5/GLM-5.2 MTP compatibility enhancements | Enhanced model compatibility and process-pool serialization capabilities for scenarios combining repetition, MTP, and torch.compile. Restored parallel search capability for throughput_optimizer. |
| msModeling | throughput_optimizer result analysis & token budget adaptation | Added support for auto mode for --max-batched-tokens, attempting token budgets in order of 4×, 2×, and 1× input_length, with automatic fallback on Prefill OOM, reducing case construction and tuning overhead. |
| msModeling | EvalScope benchmark plugin | Added EvalScope as a supplementary benchmark option for serving tuning, extending model type coverage, evaluation capabilities, and ecosystem integration to support tuning and result validation for serving deployment. |
| msModeling | optix optimizer engineering stability enhancements | Introduced structured loguru logging, domain-specific exception hierarchy, fail-fast validation before benchmark startup, and safety guardrails to improve serving tuning stability and troubleshooting efficiency. |
| msModeling | Build entry & dependency bootstrapping optimization | Added a unified build.py at the root directory as the entry point for build/test workflows. Removed strong dependency on pydantic at import-time and enabled non-interactive self-bootstrap mechanism when uv is missing, lowering installation and testing barriers. |
| msModeling | CI Gate & test entry stability enhancements | Updated build.py test to run the full pytest suite by default when MSMODELING_TEST_MAP_PATH is not set. When the variable is configured, the CI Gate incremental test mode is enabled, enhancing regression protection capabilities. |
| msModeling | Pre-commit secret scanning workflow enhancement | Added a local offline gitleaks-offline-scan hook to scan staged files for sensitive information before commits, reducing the risk of accidental secret exposure. |
| msModeling | Documentation & model support clarification improvements | Fixed broken links, Markdown formatting issues, and missing model support notes. Improved documentation usability for high-frequency scenarios such as installation, running, and result interpretation. |
5. Change Description
| Tool | Change | Impact |
|---|---|---|
| msKPP | Documentation restructuring and refinement | Conducted a large-scale documentation usability overhaul. Fixed correctness issues in the home page, installation guide, and quick-start guide; corrected typos in the LICENSE statement; optimized the description of SoC type retrieval; and merges documentation. |
| msKPP | Clean code configuration tuning | Optimized C++ clean code check configurations and adjusted code style guidelines to improve overall codebase consistency. |
| msOpGen | Submodule update strategy optimization | Modified submodule download logic to always fetch the latest submodule information instead of pinning to an old version so that operator project templates remain synchronized with the most up-to-date codebase. |
| msOpGen | Documentation restructuring and refinement | Conducted a large-scale documentation usability overhaul. Fixed broken jump links pointing to the master branch, added installation risk warnings, optimized the quick-start guide and developer guide, merged documentation, and added a new MatmulLeakyRelu custom operator case study. |
| msDebug | DisassemblerHelper class refactoring | Refactored the disassembly helper class to support various chip models, unifying instruction determination logic. No direct impact on upper-layer debugging interfaces, but internal instruction decision behavior has been adjusted. |
| msDebug | Pre-commit configuration enhancement | Added clang-tidy hook and code check configuration files to raise the bar for code merge quality. Developers must pass clang-tidy checks before local commits. |
| msDebug | Version display improvement | Added msDebug version number in the tool startup welcome banner, allowing users to quickly verify the current tool version. |
| msSanitizer | Contention detection pipe replay capability refactoring | Extracted the PipelineReplayer class as a shared utility for reuse across other detection flows, such as inter-rank contention detection. External interface behavior remains unchanged, but internal architecture has been refactored—users with custom detection pipelines are advised to review the changes. |
| msSanitizer | Memory detection performance optimization | Optimized detection latency by reducing unnecessary instruction recording and filtering logic, improving efficiency in large-scale operator scenarios. |
| msSanitizer | Removal of retired deliverables | Cleaned up build scripts and code for retired deliverables to streamline the repository structure and reduce maintenance overhead. Users relying on older deliverables must follow the new build process. |
| msSanitizer | Msopscommon submodule upgrade | Upgraded the submodule to be compatible with CMake 4.x, aligning with the CMake version requirement of the unified build entry. |
| msSanitizer | Atomic instruction instrumentation mode enhancement | Added instrumentation support for atomic mode, requiring the --check-level parameter. Detection accuracy is improved for scenarios involving atomic instructions. |
| msOpProf | Removal of theoretical stall cycle (TheoStallCyc ) feature | Breaking change: Removed the TheoStallCyc statistics featur. Related command-line parameters and output fields are no longer available. Users relying on this feature should use Top Stall Reason analysis. |
| msOpProf | Timeline parameter adjustments | Modified timeline-related parameter names and field formats. Existing scripts using old parameter names must be updated accordingly. |
| msOpProf | Instrumentation module class relationship refactoring | Refactored class relationships within the instrumentation module, extracted common methods, and removed redundant code to improve maintainability. External interface behavior is expected to remain unchanged. |
| msOpProf | --soc-version for model specification | --soc-version can now be used to explicitly specify the chip model, avoiding inaccuracies from automatic detection. Typical use cases and scenarios are updated in the documentation accordingly. |
| msOpProf | Pre-commit check enhancements | Added clang-tidy and pre-commit configuration files; removed Python code checks with only C++ checks retained. Developers are now required to pass clang-tidy checks before local commits. |
| msKL | Documentation restructuring and refinement | Conducted a large-scale documentation usability overhaul. Fixed errors in the uninstall command within the installation guide, corrected documentation inaccuracies, merged documentation, and updated materials for improved usability. |
| msKL | Interface compatibility scanning | Introduced interface compatibility scanning tests to ensure backward compatibility of Python APIs. |
| msTX | CMake 4.x compatibility adaptation | Fixed a compatibility issue where the -pie option is only applicable to executables under CMake 4.x, ensuring successful compilation with newer CMake versions. |
| msTX | Security statement supplement | Added risk warnings in the documentation regarding the use of LD_LIBRARY_PATH to load .so files, reminding users to ensure secure loading paths. |
| msTX | Documentation restructuring and refinement | Conducted a large-scale documentation usability overhaul, including README optimization and AIDD scanning improvements. |
| msProf | New build scheme and unified build entry | Adapted to the new unified build image and build infrastructure. Unified version numbering, reserved extension parameters, and standardized output paths. Addressed dynamic linking issues for packages built with the new image, and implemented compatibility modifications for both new and old images. Automation scripts that depend on the legacy build process must be updated accordingly. |
| msProf | Security policy simplification | Relaxed several security restrictions to improve usability and reduce unnecessary blocking: changed symlink validation to warnings, converted permission/owner anomalies to warnings, extended path length limit to 4096, relaxed file read restrictions, and loosen Parser size limits. |
| msProf | HCCL dataSize handling adjustment | For HCCL data where dataSize exceeds U32Max, the value is now uniformly displayed as invalid and excluded from bandwidth calculations. Breaking Change: This fix will be reverted in a later update; the default value for HCCL will change to 0 instead of U32Max. Parsing scripts that depend on the dataSize field should verify compatibility accordingly. |
| msProf | DPU column naming update | Renamed DPU data column to "CANN DPU" to align with the design specification. |
| msProf | C-style constants consolidation | Consolidated duplicated constants previously scattered across anonymous namespaces in various C++ files into a unified common_constant.h. Merged default_value_constant, time_unit_constant, unified_db_constant, and other related files. Migrated msprof_db related files from the viewer directory to application/database. No business logic changes are involved. |
| msProbe | New build scheme and unified build entry | Adapted to the new unified build image and build project structure. Unified version numbering and reserved extension parameters, and standardized output paths. |
| msProbe | Security policy simplification | Relaxed several security restrictions to improve usability and reduce unnecessary blocking: changed symlink validation to warnings, converted permission/owner anomalies to warnings, extended path length limit to 4096, relaxed file read restrictions, and loosen Parser size limits. |
| msProbe | msProbe ATB data dump readability improvement | Added support for preserving the concrete input/output tensor names of operators in ATB data dump scenarios, facilitating comparison with OM-scenario data. |
| msProbe | Quick start guide optimization | Revised the quick-start documentation so that command-line sections fully meet a copy/paste execution standard, and resolved environment issues through containerization. |
| msMonitor | Security policy streamlining | Changed symlink and invalid character validation to warnings only, without interrupting the workflow; removed directory readable/writable checks, delegating to OS mechanisms; removed permission control for file creation; disabled TLS certificate validation by default. These changes remove blocking in Docker containers, shared clusters, NFS mounts, and similar environments, greatly improving usability. To enable RPC TLS encryption, explicitly configure the certificate directory. |
| msMonitor | Unified build entry and build image upgrade | Breaking change: Introduced build.py as the unified build entry, centralizing output paths and submodule management under build.py. The build image is upgraded from openEuler 22 to 24.03. For dynolog_npu, added _GLIBCXX_USE_CXX11_ABI=0 to ensure compatibility with the bundled protobuf; OpenSSL is minimized with no-legacy no-engine, and packaging now uses fakeroot. Existing scripts relying on the old scripts/build.sh update_and_checkout_submodule() flow or secondary development based on the openEuler 22 image need to be adapted accordingly. |
| msMonitor | whl package version control via environment variable | Extended setup.py to support specifying the wheel package version via the WHL_VERSION environment variable, simplifying version injection during the packaging process. |
| msMonitor | Documentation structure and installation guide overhaul | Restructured the msMonitor documentation directory: reorganized the installation guide with a clear main flow covering package installation, source compilation, upgrade, uninstallation, and logs, supplemented with installation validation, historical version clone instructions, and upgrade compatibility notes. Added descriptions for the dyno status command. Completed the documentation of the version 26.0.0 and fixed link issues. External references relying on the old documentation paths need to be updated accordingly. |
| msPTI | Kernel type retrieval adjustment | Changed the task type field in task track data to be read from the type hash instead of being restricted to specific enum values, adapting to Runtime enum changes. Downstream parsing logic that depends on the old enum mapping should be updated accordingly. |
| msPTI | Unified build versioning and adaptation to new build scheme | CI now uniformly sets whl and run package versions via WHL_VERSION/BUILD_VERSION environment variables, and adapts to the unified build image and project structure with consolidated entry and output paths. The whl package version is no longer hardcoded to 0.0.1; automation workflows relying on the old version scheme must be updated accordingly. |
| msServiceProfiler | Unified build entry and output path | Adapted to the new unified build image by consolidating build entry scripts and output paths. No functional impact, but automation workflows that depend on old build script paths need to be updated accordingly. |
| msServiceProfiler | Package naming refresh based on tags | Run and whl package names now support automatic refresh based on tag names. Deployment or download scripts that rely on old package names should verify and adjust to the naming changes. |
| msServiceProfiler | ms_service_metric whl package | The build_whl.sh script now additionally builds the ms_service_metric whl package alongside the existing ms_service_profiler whl. To use online monitoring capabilities, this new package must be installed. |
| msServiceProfiler | tzdata dependency | Added tzdata to serving Python dependencies to resolve timezone parsing errors (e.g., Asia/Shanghai) in some environments. Offline installation setups must also include this dependency. |
| msServiceProfiler | A5 series product name unification | Corrected product type naming for the A5 series across documentation; updated installation guide accordingly. References using legacy names in existing materials should be updated accordingly. |
| msServiceProfiler | Enhanced vLLM service process management | Strengthened the logic for terminating vLLM services; changed timeout scenarios from errors to warnings to reduce interruptions caused by process residues or timeouts. |
| msServiceProfiler | Pre-commit public check rollout | Deployed pre-commit functionality across the entire repository to unify code style and static check baselines. Contributors are now required to pass pre-commit checks before submitting PRs. |
| msprof-analyze | GE auto-fusion performance data collection adjustment | The auto-fusion profile data collection for TensorFlow scenarios has been migrated from PTA to acl/ge/msTX markers, using the msprof command for data collection. Users of autofuse_performance_comparison must follow the new collection workflow. |
| msprof-analyze | Removal of MFU output from module_statistic | Breaking change: Removed MFU calculation and avgMFU output from module_statistic; these are now provided by operator_mfurecipe at both kernel and module levels. Scripts that depend on the MFU fields in module_statistic must switch to operator_mfu. |
| msprof-analyze | Third-party dependency version constraint adjustments | Removed upper version caps for numpy and pandas, replacing them with >= constraints (numpy>=1.26.4, pandas>=2.3.2) to ensure compatibility with mainstream ecosystems including PyTorch, vLLM, SGLang, Triton, Megatron-LM, and verl. |
| msprof-analyze | Packaging content normalization | The misc and test directories are no longer included in the built wheel. After installation, the site-packages directory will no longer contain an extraneous misc folder, preventing accidental imports like from misc.inductor_triton_performance_comparison. Historical installations must remove the residual site-packages/misc directory. |
| msprof-analyze | Cluster analysis data simplification deprecation | Cluster analysis now permanently enables data_simplification; dead code such as CommunicationDBGroup that is no longer invoked has been removed. This change affects internal implementation paths only and does not impact user analysis results. |
| msModelSlim | Directory restructuring: merged feature descriptions, model support list, algorithm explanations, and format specifications in the user guide | Centralizes reference paths and improves search efficiency. |
| msModelSlim | Dependency precheck log level adjustment from error to warning with version dependency info reported only when adapter errors occur | Reduces precheck interruptions and improves quantization workflow continuity. |
| msModelSlim | Introduced unified build entry supporting both build and test; removed legacy build entry | Streamlines the build process and enhances maintainability. |
6. Resolved Issues
| Tool | Fix | Scope |
|---|---|---|
| msKPP | Fixed compilation error caused by missing header file of the uint32_t type. | C++ compilation scenarios |
| msKPP | Fixed installation usability issues caused by permission and symlink validation, along with security vulnerability fixes in msKPP. | Installation/deployment and security compliance scenarios |
| msKPP | Fixed security issues in code based on Chongming security scans. | Security scanning scenarios |
| msOpGen | Fixed test_create_cmd_for_mindspore execution failure under Python 3.11. | Python 3.11 development environments |
| msOpGen | Fixed installation usability issues related to permission and symlink validation. User-friendly prompts will be provided instead of direct errors when path permissions are insufficient. | Installation/deployment scenarios |
| msOpGen | Fixed security vulnerabilities in msOpGen, including issues identified by Chongming security scans. | Security compliance scenarios |
| msDebug | Fixed multiple host/device breakpoint issues: garbled display for host-side uint8_t* pointers, failure to hit device-side breakpoints when both host and device breakpoints exist and are canceled, improper suppression of host breakpoints when dual breakpoints exist on the same line; added timeout wait to improve breakpoint stability. | Host/Device hybrid breakpoint debugging scenarios |
| msDebug | Fixed multiple coredump parsing issues for Ascend 950PR/DT: inaccurate main scalar call stack, abnormal call stack parsing, invalid error info register numbers, etc. | Coredump parsing scenarios on Ascend 950PR/DT chips |
| msDebug | Fixed breakpoint failure when ASCEND_RT_VISIBLE_DEVICES environment variable is used to set virtual device IDs on Ascend 950PR/DT chips. | Multi-card debugging scenarios on Ascend 950PR/DT chips |
| msDebug | Fixed an issue where step-over execution was excessively slow in multi-core operator scenarios, and where step-over used an incorrect instruction evaluator leading to abnormal behavior | Single-step debugging scenarios for multi-core operators |
| msDebug | Fixed incorrect register table display on A2 chips. | Register display scenarios on A2 chips |
| msDebug | Fixed Ctrl-C interrupt functionality to restore the use's ability to terminate debug sessions early via keyboard interrupt. | Interactive debug session scenarios |
| msDebug | Fixed an issue where the global map of stub functions was uninitialized, causing exceptions. | Stub-related debug scenarios |
| msDebug | Fixed an issue preventing the msdebug-mi tool from functioning correctly. | MI interface debug scenarios |
| msDebug | Fixed compilation errors when using the Clang compiler. | Clang compiler scenarios |
| msDebug | Fixed multiple security vulnerabilities, including dependency updates for CVE fixes and security hardening for msDebug usability. | Security compliance scenarios |
| msDebug | Fixed usability issues caused by permission and symlink verification, as well as device synchronization timeouts in multi-rank debugging. | Installation/deployment and multi-rank debug scenarios |
| msDebug | Fixed a null pointer dereference issue in log printing to avoid potential crash risks. | Log output scenarios |
| msSanitizer | Fixed false positives in LocalTensor out-of-bounds detection that incorrectly reported violations in certain AscendC API scenarios. | LocalTensor memory out-of-bounds detection scenarios |
| msSanitizer | Fixed false negatives in race conditions between pipe-s pipeline and other pipeline, and false positives in shmem repository mix-operator races. | Inter-pipeline race detection and shmem operator scenarios |
| msSanitizer | Fixed multiple SIMT detection issues: false positives for single-thread races, incorrect UB max range modeling, simt_callee function call restrictions, --block-id invalidation causing partial block exception output loss, coredump errors after compiler stub function support, incorrect kernel name display, entry instruction parsing order issues, and failures in some uninitialized detection scenarios. | SIMT architecture detection scenarios |
| msSanitizer | Fixed cross-rank race detection being incorrectly triggered when --check-cross-npu-races is enabled outside racecheck scenarios, and corrected filtering logic for race detection algorithms under dynamic instrumentation. | Cross-rank race detection scenarios |
| msSanitizer | Fixed false positives caused by not skipping A2 dynamic instrumentation operators during cross-rank race detection. | A2 chip cross-rank race detection |
| msSanitizer | Fixed instrumentation errors caused by shared instrType in set_lrelu_alpha dynamic stubs, and resolved false illegal free reports due to BYPASS memory records participating in memory detection. | Dynamic instrumentation and memory detection scenarios |
| msSanitizer | Fixed memory leaks not being detected when --log-level=error and --leak-check=yes were used together. | Memory leak detection scenarios |
| msSanitizer | Fixed false competition warnings caused by missing AtomicMode checks in single-pipeline competition detection. | Single-pipeline competition detection scenarios |
| msSanitizer | Fixed protocol parsing errors caused by numeric wraparound. | Kernel protocol parsing scenarios |
| msSanitizer | Fixed potential OOM issues due to rankDim overflow in mstxHcclRecord. | Multi-card communication recording scenarios |
| msSanitizer | Fixed compilation errors caused by moving temporary objects that prevented compiler optimizations. | Specific compilation scenarios |
| msSanitizer | Fixed hang issues when detecting AICPU operators. | AICPU operator detection scenarios |
| msSanitizer | Removed the -fno-jump-tables compilation flag for A5 to resolve related compatibility issues. | A5 chip compilation scenarios |
| msSanitizer | Fixed false positives in online detection involving atomic instructions, context management errors in multi-process single-rank sharing scenarios, usability issues caused by permission and symlink validation, and multiple security issues identified by Chongming scans. | Online detection, multi-process scenarios, and security |
| msOpProf | Fixed incomplete pipeline graph data caused by abnormal dynamic instrumentation of cube operators. | Cube operator performance analysis scenarios |
| msOpProf | Fixed endMark marker issues: added latency to ensure correct buffer input to the bus, fixed tail truncation and continuation flags, and restored integrity of timeline tail data. | Timeline data collection scenarios |
| msOpProf | Fixed abnormal display of VEC unit memory heatmaps and corrected related documentation descriptions accordingly. | VEC unit memory heatmap scenarios |
| msOpProf | Fixed abnormal bandwidth values when calculating bandwidth rates on certain chips. | Bandwidth rate statistics scenarios |
| msOpProf | Fixed the restriction that simt_callee functions could only call other simt_callee functions. | SIMT operator tuning scenarios |
| msOpProf | Fixed sampling result deviation caused by incorrect PC Sampling formula calculations. | PC Sampling performance sampling scenarios |
| msOpProf | Fixed timeline misalignment between warps caused by unaligned timestamps in Warp Timeline. | Warp Timeline display scenarios |
| msOpProf | Fixed application mode pipetimeline runtime failures. | Application mode pipeline timeline scenarios |
| msOpProf | Fixed obscure errors when kernel-name fails to match, with user-friendly hints added; resolved log level inconsistencies between debug and info logs. | Logging and error messaging scenarios |
| msOpProf | Fixed various logging errors and parameter validation issues caused by chip-isolated parameters. | Log output and parameter validation scenarios |
| msOpProf | Fixed multiple security vulnerabilities and usability issues, including permission/symlink validation and code security fixes. | Security compliance and installation/deployment scenarios |
| msOpProf | Fixed simulation dump folder instr log abnormal termination, missing libprofplugin_instrprofstart.so artifacts, compilation failures due to missing headers, and SIMT cube compilation optimizations, among other build and runtime issues. | Build, runtime, and data export scenarios |
| msKL | Fixed multiple security and usability issues, including security skill code scan findings and vulnerabilities. | Security compliance and installation/deployment scenarios |
| msKL | Fixed security issues in code based on Chongming security scans. | Security scanning scenarios |
| msTX | Fixed import failure for mstx due to the missing msTX module when the msTX wheel package is installed standalone. | Wheel package installation scenarios |
| msTX | Fixed prof msTX functionality anomalies in Python environments. | Python Profiler integration scenarios |
| msTX | Fixed source code links not pointing to the correct version. | Documentation and link reference scenarios |
| msTX | Fixed security issues in code based on Chongming security scans. | Security scanning scenarios |
| msProf | Fixed missing communication large operators in timelines when msprof collects ACLGraph multi-rank data. | ACLGraph multi-rank communication data collection and parsing scenarios |
| msProf | Fixed missing SIMT operators in timelines when msprof collects SIMT operator data, and added support for parsing additional profiling info reported by launchKernel. | SIMT operator collection and visualization scenarios |
| msProf | Fixed zero-value output for Acc PMU and BIU data collected by msprof on A3 servers. | A3 server PMU/BIU collection scenarios |
| msProf | Fixed missing npu_module_mem.csv file when profile_memory is enabled. | Memory data collection scenarios |
| msProf | Fixed stars data parsing anomalies, corrected log parser header-tail matching logic (fixing mis-matches in start1/start2/end2 scenarios), and added 0x6bd3 magic number validation to filter invalid data. | Stars data parsing scenarios |
| msProf | Fixed UB and DPU data issues when exporting DB table data on new chips, including incorrect deviceId handling logic in the UB parser and DPU data not being parsed or persisted | New chip DB export scenarios |
| msProf | Silenced platform sub-package installation details printed to screen during run package installation. | Run package installation scenarios |
| msProf | Fixed platform_profiler.run installation failure for non-root users due to 550 permissions on lib64 and profiler directories during full CANN package installation. | CANN full package installation/uninstallation scenarios |
| msProf | Fixed missing operator shape and dtype information for mixed streams in offline inference scripts. | Offline inference mixed-stream collection scenarios |
| msProf | Fixed incorrect device instrumentation timing in ACLGraph scenarios; adapted to mark/range tag ID changes after CANN 9.1.0 and corrected pairing logic for duplicate index_id values. | ACLGraph scenario msTX instrumentation parsing |
| msProf | Fixed script execution failure in core binding due to incorrect file path settings causing import errors. | CPU affinity binding scenarios |
| msProf | Fixed missing UB→GM data when using msprof -op on Ascend 950PR/DT products. | Ascend 950PR/DT chip UB data collection scenarios |
| msProf | Fixed --application causing "Script params are invalid" errors. | Command-line argument parsing scenarios |
| msProf | Fixed missing chrome_tracing.json in output after data collection and parsing. | Collection and parsing output scenarios |
| msProf | Fixed security vulnerabilities in platform-profiler.run caused by SO files not complying with secure compilation options. | Secure compilation scenarios |
| msProf | Added security notes for strace and gil_tracer tools in the security statement documentation. | Security statement documentation scenarios |
| msProf | Fixed task overflow caused by refreshing task ID during set batch id, which resulted in loss of high 16-bit data. | Task ID calculation scenarios |
| msProf | Fixed DB data index anomalies and kfcTask loss caused by multiple communication operator types sharing the same communication group; now index for AICPU-type operators is computed separately . | Communication operator DB data processing scenarios |
| msProf | Corrected CCU task ID to 32-bit, fixed stream ID default value, and adjusted Host CCU additional data matching with device CCU. | CCU data parsing scenarios |
| msProf | Fixed regex escape character warnings when installing the run package on grep 3.8+. | Run package installation compatibility scenarios |
| msProf | Fixed installation failure of platform_profiler.run due to undefined PLATFORM_PROFILER_RUN variable. | Run package installation scenarios |
| msProf | Adapted the platform_profiler.run packaging flow to bundle it into the msprof run package and install it into the CANN package; synchronised the uninstall script accordingly. | Run package packaging and installation scenarios |
| msMonitor | Fixed a crash in msMonitor when processing msPTI collected data, caused by direct C-style const char* to std::string conversion without null-pointer checks. Added a unified SafeCstrToString validation wrapper and replaced all affected call scenarios. | msPTI data processing scenarios |
| msMonitor | Fixed npu_monitor not clearing the duration value after stop, causing the next collection (with no duration specified) to reuse the previous value and stop prematurely. Added proper duration reset logic. | npu-monitor/nputrace duration-controlled collection scenarios |
| msMonitor | Fixed dyno binaries failing security compilation options (SP stack protection) scans; reinforced build scripts and compiler options to pass secbinarycheck security scans. | dyno/dynolog release package security scanning scenarios |
| msMonitor | Fixed installation failure in offline environments due to undeclared xlsxwriter dependency; clarified dependency notes in installation guide and included them via wheel package installation. | Offline package installation scenarios |
| msPTI | Fixed a race condition vulnerability in CallbackManager between ExecuteCallback and UnInit when executed concurrently; replaced subscriber_ptr_ from unique_ptr to shared_ptr, applied C++11 atomic_load/atomic_exchange, and serialized Init/UnInit with a mutex to prevent process crashes during concurrent callback and uninit calls. | Concurrent callback subscription and uninit execution scenarios (A2/A3) |
| msPTI | Fixed timestamp calculation errors for marker data when host freq retrieval fails: corrected ContextManager's device timestamp logic, and fixed the branch in msTX that returns either mono raw or syscnt timestamps based on the host freq enable flag. | msTX marker data collection in scenarios where host freq is unsupported (Ascend 950/Debian) |
| msPTI | Fixed msPTI kernel data collection failures caused by Runtime kernel type enum changes: task type is now derived from type hash instead of hardcoded enum values, ensuring compatibility across Runtime upgrades. | Kernel data collection scenarios (A2/A3/Ascend 950) |
| msPTI | Fixed incomplete data collection (only Communication type) when running msPTI sample sets on Ascend 950: introduced CommunicationMonitor and the Ascend 950 data flush mechanism to restore full data collection. | Ascend 950 sample set execution and data collection scenarios |
| msPTI | Fixed execution errors in some samples due to deviceId being set to 4 when the device does not exist; changed default deviceId to 0 for all samples. | Sample execution scenarios |
| msPTI | Fixed --check error in the run package by adding proper handling and immediate exit after --check. | Run package installation validation scenarios |
| msPTI | Fixed data loss of hash/type information reported by some operators before operator execution due to missing registration in msPTI; also fixed compilation errors under high-version GCC (e.g., GCC 13). | Hash/type data collection and source compilation scenarios |
| msPTI | Fixed broken links in the home-page README and revamped it according to documentation usability requirements. | Documentation browsing scenarios |
| msPTI | Corrected return value descriptions for msptiActivityPushExternalCorrelationId and msptiActivityPopExternalCorrelationId: push only returns success, while pop may return MSPTI_ERROR_INVALID_PARAMETER and MSPTI_ERROR_QUEUE_EMPTY. | API documentation and usage scenarios |
| msServiceProfiler | Fixed potential command injection risks in subprocess.run() calls within compare.py and msprof_data_source.py, and unified path argument escaping. | Security scanning and command execution scenarios |
| msServiceProfiler | Removed sensitive information (default passwords) from the METADATA in release packages and completed security hardening. | Release package security compliance |
| msServiceProfiler | Fixed a security issue where relative library names caused untrusted search paths (DT_RPATH). | Dynamic library loading security |
| msServiceProfiler | Restricted unsafe dynamic execution paths by tightening configuration-driven dynamic imports and expression evaluation risks. | Configuration loading security |
| msServiceProfiler | Fixed a time-of-check to time-of-use (TOCTOU) race condition caused by ms_open file status caching. | File operation security |
| msServiceProfiler | Fixed package permission changes introduced by serving installation. | Installation and deployment scenarios |
| msServiceProfiler | Fixed global vLLM metrics being polluted by the phase label causing duplicate statistics; set the global phase attribute to "all" for static metrics. | ms-service-metric statistics scenarios |
| msServiceProfiler | Fixed incorrect first_token_latency (TTFT) calculation by correcting the second token latency hook point, ensuring TTFT accurately reflects the duration from request arrival to batch execution completion. | vLLM latency parsing scenarios |
| msServiceProfiler | Fixed timezone errors caused by missing the Asia/Shanghai timezone definition during data parsing by adding tzdata as a dependency. | Data parsing scenarios |
| msServiceProfiler | Fixed hook point failures caused by SGLang version upgrades (from 0.5.4 to 0.5.6), restoring collection capability. | SGLang collection scenarios |
| msServiceProfiler | Fixed operator collection failure with newer CANN packages caused by invalid prof-api call interfacesl. | Operator collection scenarios |
| msServiceProfiler | Restored compatibility with the legacy device ID retrieval interface to address device identification failures from interface changes. | Profiling collection scenarios |
| msServiceProfiler | Fixed model execution collection missing in vLLM scenarios and timing inaccuracies in asynchronous scheduling. | vLLM collection scenarios |
| msServiceProfiler | Fixed residual files after run package uninstallation and improved the uninstallation workflow. | Run package installation/uninstallation scenarios |
| msServiceProfiler | Fixed unintended deletion of files under python/site-packages/bin/ during run package installation. | Run package installation scenarios |
| msServiceProfiler | Fixed handler override issue in ms-service-metric. | ms-service-metric collection scenarios |
| msServiceProfiler | Fixed kvcache metric type and dp label default value issues in vLLM scenarios. | ms-service-metric metric scenarios |
| msServiceProfiler | Fixed invalid msg records generated by Trace data under error conditions. | Trace data monitoring scenarios |
| msServiceProfiler | Fixed grep regex warnings when installing the run package in high-version grep environments, and updated the makeself third-party dependency. | Run package installation scenarios |
| msServiceProfiler | Fixed compatibility issues introduced by SGLang upgrade from v0.5.4 to v0.5.6 | SGLang compatibility scenarios |
| msprof-analyze | Fixed No module named 'misc.inductor_triton_performance_comparison' error after source build installation, caused by the misc/test directory being incorrectly packaged into the wheel. | Source build installation scenarios |
| msprof-analyze | Fixed FA ratio reporting 0 in inference scenarios by adding adapter support for the inference FA operator fusedinferattentionscore. | Inference FA statistics scenarios |
| msprof-analyze | Fixed cluster analysis failures and missing cluster_analysis_output generation when import paths contain Chinese square brackets 【 or 】. | Cluster analysis with paths containing Chinese brackets |
| msprof-analyze | Fixed No such option '-t' error when running msprof-analyze advisor all -tv to specify PyTorch version; added --torch_version / -tv option for advisor all. | advisor all command-line interface |
| msprof-analyze | Optimized LCS implementation in performance comparison by replacing custom bitmap algorithm with difflib; reduced execution time from over 7 hours to approximately 50 minutes for large datasets. | Large-scale performance comparison scenarios |
| msprof-analyze | Fixed usability issues causing by validation: changed file owner/others writable checks from blocking to warnings; made --force effective via initializer in recipe subprocesses; increased DB size limit from 8 GB to 20 GB; no longer modifies permissions when attaching to existing DB. | Validation and DB scenarios |
| msprof-analyze | Streamlined security policies: replaced hard exceptions with warning logs for symlink, long-path, and special-character validation; only abort on critical errors such as file non-existence or unreadability. | Path validation scenarios |
| msprof-analyze | Removed unnecessary os.chmod() calls after file creation to avoid overriding permission policies set by users or administrators in shared directories or container environments. | Multi-user collaboration and container scenarios |
| msprof-analyze | Removed upper version caps for numpy and pandas to resolve dependency conflicts when used alongside ecosystem components such as PyTorch, vLLM, SGLang, Triton, Megatron-LM, and verl. | Third-party dependency compatibility |
| msprof-analyze | Fixed multiple issues in cluster analysis and expert recommendations for Ascend 950PR/DT scenarios: cluster analysis closing stdout causing subsequent advisor errors due to progress bar issues; missing abstract method in SlowLinkExport; ai_core_performance_checker failing to handle empty strings when converting shape via int; memory calculation omissions in multi-output scenarios; and excessive log spamming. | Ascend 950PR/DT data collection and analysis scenarios |
| msprof-analyze | Fixed usability and correctness issues in documentation, including consistency improvements to advanced_features/README.md, added notes that communication matrix data analysis is not supported for Ascend 950PR/DT CCU scenarios in advisor_instruct and cluster_analyse_instruct, and restructured directory layout and links. | Documentation usage |
| msModelSlim | Fixed index out-of-bounds error in DeepSeek-V3 MTP. | Users leveraging MTP with DeepSeek-V3 |
| msModelSlim | Resolved OOM issue in GLM5.1 MXFP4 quantization. | Users applying MXFP4 quantization to GLM5.1 |
| msModelSlim | Fixed NaN results from all-zero blocks in MXFP8 quantization. | Users utilizing MXFP8 quantization |
| msModeling | Fixed compatibility and process-pool serialization issues in GLM-5/GLM-5.2 when repetition, MTP, and torch.compile are enabled simultaneously; restored parallel search capability. | GLM-5/GLM-5.2 service-oriented tuning scenarios |
| msModeling | Fixed runtime errors caused by GLM5 DSA contract mismatches by raising the minimum required transformers version to avoid incompatible return values from older releases | GLM5 series model emulation scenarios |
| msModeling | Fixed broken links, Markdown formatting issues, and missing support descriptions for certain models to improve documentation readability and maintainability. | Documentation reading and maintenance scenarios |
| msModeling | Refactored optix optimizer reliability with structured logging, domain-specific exceptions, pre-benchmark fail-fast validation, and safe cleanup guards. | optix serving benchmarking/tuning scenarios |
| msProbe | Fixed exception in get_relative_err caused by computing zero_mask before type conversion. | Accuracy comparison relative-error calculation |
| msProbe | Fixed parameter gradients not being reset across multiple steps. | Parameter gradient collection and monitoring |
| msProbe | Fixed broken image links in official community group documentation. | Official community group documentation display |
| msProbe | Fixed failure when converting numpy.str_ type arrays to Torch Tensors. | NumPy string array conversion functionality |
| msProbe | Fixed error reporting in HiFP8 data collection. | HiFP8 data collection functionality |
| msProbe | Fixed _merge_params raising TypeError when graph parameter metrics are None. | Graph parameter metric merging functionality |
| msProbe | Skipped NZ-format Tensor collection to prevent abnormal memory allocation. | NZ-format Tensor data collection functionality |
| msProbe | Fixed Chinese encoding anomalies in Excel files. | Excel report export functionality |
| msProbe | Upgraded frontend dependencies to address known CVE vulnerabilities. | Frontend component security |
| msProbe | Fixed Dump data failure when selective activation recomputation is enabled. | Data Dump functionality in selective activation recomputation scenarios |
| msProbe | Fixed import errors or segmentation faults when importing tool modules in environments with TensorFlow installed. | Tool module import functionality in TensorFlow environments |
7. Known Issues
| Tool | Content |
|---|---|
| msMemScope | In the multi-rank training scenario, the number of kernel_start entries in the data collected with --level=1 --events=launch is incorrect; duplicate start records for certain operators occur randomly. |
8. More Resources
For more release notes-related content, refer to the following table.
| Tool | Access Link |
|---|---|
| msKPP | Release Notes |
| msOpGen | Release Notes |
| msDebug | Release Notes |
| msSanitizer | Release Notes |
| msOpProf | Release Notes |
| msKL | Release Notes |
| msTX | Release Notes |
| msProf | Release Notes |
| msProbe | Release Notes |
| msMonitor | Release Notes |
| msPTI | Release Notes |
| msMemScope | Release Notes |
| msServiceProfiler | Release Notes |
| msInsight | Release Notes |
| msprof-analyze | Release Notes |
| msModelSlim | Release Notes |
| msModeling | Release Notes |