Installing ATB Models

This section describes how to obtain the ATB Models software package and install it. Currently, an independent ATB Models software package is not provided. Therefore, you need to obtain the software package from the MindIE image.

Procedure

Visit the Ascend image repository, where you can download the MindIE image.
Click the login button in the upper right corner of the page and log in with your Huawei account. (If you do not have a Huawei account, register one first.)
On the Image Version tab page of the MindIE image download page, find the image to download based on your device model and click Download (which is in the Operation column) on the right of the image you are going to download.
Download the image according to the displayed image download guide.
Create a decompression directory (for example, /home/{User name}/Package).
```
mkdir /home/{User name}/Package
```
Grant the read and write permissions on the path.
```
chmod u+rw /home/{User name}/Package
```
Upload the obtained ATB Models software package Ascend-mindie-atb-models_{version}_linux-{arch}_pyxxx_torchx.x.x-{abi}.tar.gz to the directory. The ATB Models software package is stored in the /opt/package directory of the MindIE image package.
The ABI version of ATB Models must be the same as that used during PyTorch compilation. You can call the torch.compiled_with_cxx11_abi() API to view the ABI version.
- If False is returned, set abi=0.
- If True is returned, set abi=1.

Go to the directory where the software package is stored and decompress it. Table 1 lists the files contained in the decompressed software package.

cd /home/{User name}/Package
tar -zxf Ascend-mindie-atb-models_{version}_linux-{arch}_pyxxx_torchx.x.x-{abi}.tar.gz

**Table 1** File list
Directory	File	Description
atb_llm	common_op_builders	Source code of atb_llm
	conf
	layers
	models
	nn
	runner
	utils
-	atb_llm-{version}-py3-none-any.whl	Installation package of atb_llm
examples	convert	Sample project directory
	__init__.py
	input.jsonl
	models
	multimodal_runner.py
	README.md
	run_edge.py
	run_fa_edge.py
	run_fa.py
	run_pa.py
	server
lib	libatb_speed_core.so	Dependency SO files
	libatb_speed_models.so
	libatb_speed_operations.so
	libatb_speed_torch.so
	libatb_torch.so
	libatb_log.so
-	public_address_statement.md	Public network address statement
-	README.md	Remarks
requirements	models	Source code of requirements
requirements	requirements.txt	Source code of requirements
-	set_env.sh	Environment variable configuration file
-	setup.py	Package file
tests	modeltest	Source code of the test tool
-	version.info	Version information

Check the permission on the pip package installation path.
To prevent the error message "module not found" from being displayed after the .whl package is successfully installed, ensure that the current user has the write permission on the installation path of the pip package when pip is used to install the .whl package. You can obtain the installation path of the pip package by running pip show {Name of the existing package}. The following is an example:
```
pip show pip
```
The following information in bold is the installation path. The actual command output varies according to the actual situation.
```
Name: pip
Version: 25.1
Summary: The PyPA recommended tool for installing Python packages.
Home-page: https://pip.pypa.io/
Author:
Author-email: The pip developers <distutils-sig@python.org>
License: MIT
Location: /root/miniconda3/envs/infor/lib/python3.11/site-packages
Requires:
Required-by:
```
Install the Python package of atb_llm in the Python environment.
```
pip install atb_llm-{version}-py3-none-any.whl
```

Environment Variable Configuration

A process-level environment variable setting script is provided to automatically set environment variables. The specified environment variables automatically become invalid after the user process ends.

source /home/{User name}/Package/set_env.sh

You can also configure permanent environment variables by modifying the ~/.bashrc file. The procedure is as follows:

Run the vi ~/.bashrc command in any directory as the running user to open the .bashrc file and append the preceding lines to the file.
Run the :wq! command to save the file and exit.
Run the source ~/.bashrc command for the modification to take effect immediately.

Parent topic: Mode 2: PM Installation