Ascend NPU is a rising star in AI computing, but most training and online inference scripts are currently based on GPUs. Due to architectural differences between NPUs and GPUs, GPU-based training and online inference scripts cannot be directly used on NPUs.
MindStudio Transplant (msTransplant), a migration analysis tool, provides one-click migration of PyTorch training scripts to Ascend NPU, allowing developers to complete migration with minimal or zero code modifications. The tool offers the PyTorch Analyse function, which helps users analyze the support status of APIs, third-party library APIs, affinity API analysis, and dynamic shapes in PyTorch training scripts. It also provides two migration methods: Automatic Migration and PyTorch GPU2Ascend tool migration, which migrate GPU-based scripts to NPU-based scripts. This automated approach saves the learning cost and workload of manual script migration, significantly improving migration efficiency.
- (Recommended) Automatic Migration: Minimal modifications are required; you only need to import library code into the training script, and it can run directly on the Ascend NPU platform after migration.
- PyTorch GPU2Ascend Tool Migration: The migration process generates analysis files, allowing users to view API support analysis reports and the modifications made to the original training script during migration. It also supports migrating single-device scripts to multi-device scripts.
[object Object]
The actual execution of this tool's functions depends solely on the CPU, and the following mandatory dependencies must be installed in advance:
After the migration function is executed, if you need to run the migrated training script on the NPU, you must also install the following additional dependencies:
Install the matching version of the CANN Toolkit development suite package and the ops operator package. See the .
Configure environment variables.
After installing the CANN software, when using the CANN runtime user for compilation and runtime operations, you need to log in to the environment as the CANN runtime user and execute the
[object Object]command to set the environment variables. Here, ${install_path} is the installation directory of the CANN software, for example: /usr/local/Ascend/cann.[object Object]
- The analysis and migration tool currently supports the analysis and migration of training scripts for PyTorch versions 2.1.0, 2.6.0, 2.7.1, and 2.8.0.
- The original script must be able to run successfully in a GPU environment based on Python 3.7 or later.
- The execution logic after analysis and migration must remain consistent with that before migration.
- If the original code calls third-party libraries, adaptation issues may arise during the migration process. Before migrating the original code, users need to install the Ascend-adapted versions of the third-party libraries based on the libraries already called. For information on adapted third-party libraries and usage guides, please refer to the Ascend Extension for PyTorch and Third-party Library Support List.
- The FusedAdam optimizer used in APEX does not support migration using the automatic migration and PyTorch GPU2Ascend tools. If the original code contains this optimizer, users need to modify it themselves.
- The current analysis tool does not support affinity API analysis for native functions such as
[object Object],[object Object],[object Object],[object Object],[object Object],[object Object], or[object Object]. If the original training script involves any of the above native functions, refer to the "Python APIs > torch_npu.contrib" section in the Ascend Extension for PyTorch Custom API Reference for analysis and replacement. - If the user's training script contains the amp_C module, which is unsupported on the Ascend NPU platform, the user must manually delete the code related to
[object Object]before proceeding with training. - Since the converted script runs on a different platform than the original script, the migrated script may throw exceptions during debugging and execution due to reasons such as operator differences, causing the process to terminate. Such exceptions require further debugging and resolution by the user based on the error information.
- When using the tool in a Linux environment, for security and the principle of least privilege, this tool should not be operated using high-privilege accounts such as root. It is recommended to install and execute it with normal user permissions.
- When using the tool in a Linux environment, ensure that the umask value of the user executing the tool is greater than or equal to 0027 before use. Otherwise, the permissions on data files and directories generated by the tool may be excessively permissive.
- When using the tool in a Linux environment, users must ensure the principle of least privilege is followed. For example, files input to the tool must not be writable by other users. In scenarios with stricter security requirements, it must also be ensured that input files are not writable by group users.
- Since this tool depends on CANN, for security purposes, the CANN package installed by default under the same low-privilege user should be used. After executing the source command, do not arbitrarily modify the environment variables involved in
[object Object]. - This tool is intended for development and debugging. It is not recommended for use in production environments.
The Analysis and Migration Tool can migrate GPU-based training scripts to scripts that support NPU, significantly improving script migration speed and reducing developer workload. This sample allows developers to quickly experience the migration efficiency of Automatic Migration (recommended) and the PyTorch GPU2Ascend tool.
This sample uses the ResNet-50 model with the ImageNet dataset.
- Prepare a training server based on the Atlas training series and install the corresponding driver and firmware. For driver and firmware installation, refer to the "" section in the CANN Software Installation Guide.
- Install the development suite package Ascend-cann-toolkit and the ops operator package. For details, refer to the "" section in the CANN Software Installation Guide.
- Taking the installation of PyTorch 2.1.0 as an example, for specific operations, refer to the "Installing PyTorch" section in the Ascend Extension for PyTorch Software Installation Guide to complete the installation of the PyTorch framework and the torch_npu plugin.
- Download the file and place the obtained ResNet50 model in a user-defined path (such as
[object Object]).
With minimal modifications, you only need to import library code into the training script, and it can run directly on the Ascend NPU platform after migration.
Import the library code for automatic migration into the training script file.
[object Object]Switch the directory to the path where the migrated training script is located (using
[object Object]as an example), and run the following command to perform training with a dummy dataset. The migrated training script can run normally on the NPU.Iteration logs start printing, indicating that the training function migration is successful.
[object Object]The migration tool automatically saves the weights successfully, indicating that the migration is successful.
Go to the path where the migration tool is located.
[object Object]Execute the script migration task, and configure the information by referring to .
[object Object][object Object]is the original script path,[object Object]is the output path for the script migration result, and 2.1.0 is the PyTorch framework version of the original script.Switch the directory to the path of the training script after migration (using /home/user as an example), and execute the following command to perform training with a virtual dataset. The training script after migration can run normally on the NPU.
Iteration logs start to print, indicating that the training function migration is successful.
[object Object]After the script migration is complete, go to the output path of the script migration result to view the result files.
[object Object]
The migration tool automatically saves the weights successfully, indicating that the migration is successful.
PyTorch Analyse provides analysis scripts to help users analyze the support status of APIs, third-party library suites, affinity API analysis, and dynamic shapes in GPU-based PyTorch training scripts before performing migration operations. For details, see .
The analysis script is located at: [object Object].
Table 1 Analysis mode introduction [object Object][object Object]
"[]" indicates optional parameters, which can be omitted in actual use; "<>" indicates variables.
Table 2 Parameter description[object Object][object Object]
Navigate to the directory where the analysis tool is located.
[object Object]Start the analysis task.
Refer to for configuration information, and run the following command to start the analysis task.
[object Object]Where
[object Object]is the path of the script to be analyzed,[object Object]is the output path for the analysis result, and[object Object]is the framework version of the script to be analyzed.If the analysis mode specified by the
[object Object]parameter is dynamic_shape, after the analysis task is completed, you need to modify the training script by referring to to obtain the dynamic shape analysis report.After the analysis is complete, go to the script analysis result output path and view the analysis report. For details, see .
Go to the path where the analysis tool is located.
[object Object]Use the third-party library suite analysis function of the
[object Object]parameter to obtain the list of APIs that do not support migration in the third-party library (csv file).[object Object][object Object]is the path to the third-party library folder,[object Object]is the output path for the results, and[object Object]is the framework version of the script to be analyzed.After this command is executed, a list of APIs in the third-party library that are unsupported for migration, i.e., the
[object Object]file, is generated in the[object Object]directory.Pass the CSV file obtained in the previous step to
[object Object]to get the information about the third-party library APIs in the current training script that are unsupported for migration.[object Object][object Object]is the path to the model script folder, and[object Object]is the output path for the results.After the analysis is complete, go to the script analysis result output path and view the analysis report. For details, see .
When the analysis mode is "torch_apis", the analysis results are as follows:
[object Object]Table 3 Introduction to csv files in "torch_apis" mode
[object Object]undefined
Figure 1 Example of an unsupported API list
[object Object][object Object]
Table 4 PyTorch API interface information[object Object][object Object]
[object Object][object Object]
[object Object]When the analysis mode is "third_party", the analysis results are as follows:
[object Object]Table 5 Description of CSV files in "third_party" mode
[object Object]undefined
Figure 2 Example of an unsupported API list [object Object][object Object]
When the analysis mode is "affinity_apis", the analysis results are as follows:
[object Object]
The analysis report [object Object] includes the call information of native APIs and categorizes them into several types: class, function, torch (PyTorch framework API), and special (special expressions). Based on the analysis report, users can manually replace native APIs with the specified affinity APIs in the training script. The replaced script delivers better performance when running on Ascend AI Processors. An example of the analysis report is shown below.
Figure 3 Affinity API analysis report example
When the analysis mode is "dynamic_shape", the analysis result is as follows:
[object Object]After generating the dynamic shape analysis result file, you need to first modify the for loop that reads the training dataset in the training script file under the analysis result output directory to manually enable dynamic shape detection. Refer to the example below for modification.
Before modification:
[object Object]After modification:
[object Object]Run the modified training script after analysis, and the dynamic shape analysis report msft_dynamic_shape_analysis_report.csv will be generated in the root directory where the analysis result file is located.
[object Object]
This chapter guides users on migrating PyTorch training scripts from a GPU platform to the Ascend NPU platform. The automatic migration method supports the migration of training scripts for PyTorch versions 2.1.0, 2.6.0, 2.7.1, and 2.8.0. This method is relatively simple and requires minimal modifications, only needing the import of a library code in the training script.
- Because the automatic migration tool uses Python's dynamic features, but torch.jit.script does not support Python's dynamic syntax, conflicts will occur when using the automatic migration feature if the original training script contains torch.jit.script. Currently, the torch.jit.script feature is disabled during automatic migration. If the user script must use the torch.jit.script feature, please use the for migration.
- The automatic migration tool may have functional conflicts with third-party libraries that have been adapted for Ascend. If a conflict occurs, use the for migration.
- Currently, automatic migration does not support the channel_last feature. It is recommended that users use contiguous as a replacement.
- If the backend used in the original script is
[object Object], after initializing the process group with[object Object], the backend has been replaced by the automatic migration tool with[object Object]. If subsequent code logic includes a check on whether the backend is[object Object], such as assert[object Object]or[object Object], manually change the string[object Object]to[object Object]. - If the user's training script contains the
[object Object]interface, which is unsupported on the Ascend NPU platform, it needs to be manually changed to the[object Object]interface.
Import the library code for automatic migration.
Insert the following reference content at the first line of the training entry
[object Object]file. For example, insert the following reference content at the first line of train.py.[object Object]The migration operation is complete. Refer to and the training process provided by the original script to directly run the modified model script on the Ascend NPU platform.
After training is complete, the migration tool automatically saves the weights successfully, indicating that the migration is successful. If the migration fails, refer to for resolution.
- If the model includes evaluation or online inference functionality, you can also import the automatic migration library code in the corresponding scripts and determine whether the migration is successful by comparing the evaluation/inference results and log outputs with those on GPU or CPU.
- If some CUDA API errors are reported during training, it may be caused by unsupported APIs (operator APIs or framework APIs). You can refer to the following solutions to resolve the issue.
- Use the analysis and migration tool to analyze the model script, obtain the list of APIs with uncertain support status, and submit an ISSUE for assistance in the .
- For Ascend C operators, refer to the "PyTorch Framework Feature Guide > Custom Operator Adaptation Development > " section in for operator adaptation.
This section describes the migration methods of the PyTorch GPU2Ascend tool.
- Since the converted script runs on a different platform than the original script, exceptions may occur during debugging and running of the migrated script due to operator differences and other reasons, causing the process to terminate. Such exceptions require further debugging and resolution by the user based on the error information.
- After analysis and migration, you can perform training by referring to the training process provided by the original script.
"[]" indicates optional parameters, which can be omitted in actual use; "<>" indicates variables.
Table 6 Parameter description[object Object][object Object]
Go to the path where the migration tool is located.
[object Object]Start the migration task.
Refer to for configuration information, and run the following command to start the migration task.
[object Object]Where /home/username/fmktransplt is the original script path, /home/username/fmktransplt_output is the output path for script migration results, 2.1.0 is the original script framework version, /home/train/train.py is the entry file of the training script, and model is the target model variable name. distributed and its parameters -m and -t are specified at the end of the statement.
Reference example:
[object Object]After the script migration is complete, go to the output path of the script migration result to view the result files.
[object Object]
Refer to and the training process provided by the original script to directly run the modified model script on the Ascend NPU platform.
Successfully saving the weights indicates that the weight saving function has been migrated successfully.
After training is complete, the migration tool automatically saves the weights successfully, indicating that the migration is successful.
This section mainly introduces the configuration items that need attention when performing model migration training in special scenarios.
None.
To improve model running speed, it is recommended to enable binary operators. Refer to the "" section in the CANN Software Installation Guide to install the Toolkit development suite package and the ops operator package, and then enable it as follows:
In a single-device scenario, modify the training entry file, such as main.py, and add the following code below
[object Object].[object Object]In a multi-device scenario, if the method for launching multi-device training is
[object Object],[object Object]must be added in the main function that launches the process to enable binary operators. Otherwise, the enabling method is the same as in the single-device scenario.[object Object]
If the user's training script contains the
[object Object]interface, which is unsupported on the Ascend NPU platform, it must be manually modified to the[object Object]interface for multi-device training. Refer to for modification instructions.If the user's training script contains the amp_C module, which is unsupported on the Ascend NPU platform, the user needs to manually delete the code related to
[object Object]before proceeding with training.If the user's training script contains the
[object Object]interface, it will return a[object Object]value when running on the Ascend NPU platform after migration.[object Object]
After migration, when the
[object Object]interface runs on the Ascend NPU platform, the return value does not include the minor and major attributes. It is recommended that the user comment out the code that calls the[object Object]and[object Object]attributes.
torch.utils.data.DataLoader is a utility class in PyTorch for data loading, primarily used to divide sample data into multiple mini-batches for tasks such as training, testing, and validation. Check whether the dataset loading method in the model script uses torch.utils.data.DataLoader. The sample code is as follows:
If the [object Object] parameter is enabled during migration and you want to migrate a GPU single-device script to an NPU multi-device script, perform the following steps to obtain the result files:
[object Object]
Replace the training script statement.
Replace the
[object Object]statement in the[object Object]file generated after executing the migration command with the model's original training shell script. For example, replace[object Object]with the model training command[object Object].The
[object Object]file is as follows:[object Object]Table 7 run_distributed_npu.sh parameter description
[object Object]undefined
After replacement, execute the "run_distributed_npu.sh" file, and a log file for the specified NPU will be generated.
View the result files.
After the script migration is complete, go to the result output path to view the result files. Taking the migration of a GPU single-device script to an NPU multi-device script as an example, the result files include the following content:
[object Object]View the migrated Python script. You can see that the CUDA-side APIs in the script have been replaced with NPU-side APIs.
[object Object]
Symptom
The converted code runs without any error message, only displaying "Segmentation fault".
Cause Analysis
- Possible cause 1:
The code references TensorBoard or a third-party library that includes TensorBoard. The following are known third-party libraries that reference TensorBoard.
wandb: If this library is only used for logging, you can remove the calls to this library.
transformers: This library is deeply bound to TensorFlow and TensorBoard.
Possible cause 2:
The training script contains code that compares two 0-dimensional tensors on different devices. This comparison is currently not supported on torch_npu.
Solution
Solution for Cause 1:
Comment out the related Summary and Writer calls to avoid this error. Summary and Writer are mostly used for logging and plotting, and do not affect network execution or accuracy convergence.
Solution for Cause 2:
Add python -X faulthandler before the script launch command to print thread information, locate the specific error position, and perform pdb debugging. This helps identify whether the script contains code that compares two 0-dimensional tensors on different devices. You need to manually modify the code to perform the comparison on the same device. An example is shown below:
Before modification, comparison is performed on CPU and NPU:
[object Object]After modification, add the following information to change the comparison to be performed on NPU:
[object Object]
The issue of a referenced library not found may occur in the following three scenarios. Please troubleshoot based on the actual situation:
Symptom 1
A folder or file in the current directory or subdirectory cannot find the referenced library.
Solution 1
Simply add the parent directory of that directory to the PYTHONPATH environment variable.
Symptom 2
The unfound reference library is a package that needs to be installed via pip as specified in requirements.txt.
Solution 2
You can use pip install package_name to install it. If the installation fails, you can git clone the installation package and install it using python3 setup.py install.
Symptom 3
The unfound reference library is an installation package that needs to be downloaded and installed via git clone as specified in readme.md.
Solution 3
Please download and install it as required.
Symptom
Solution
As shown in the figure above, change [object Object] to [object Object], that is, change the variable type of the current error line to int32 to avoid the issue of the Muls operator not supporting int64.
Symptom
Error "No supported Ops kernel and engine are found for [ReduceStdV2A], optype [ReduceStdV2A]", the operator ReduceStdV2A is unsupported.
Solution
You can work around this issue by using std to calculate the standard deviation and then squaring it to get var, and calling the mean API separately to obtain the mean. For example:
Modify the code as follows:
Table 8 Common runtime errors