Build and Installation

Compilation

This document explains how to build the MindIE-LLM .whl package from source and install and run it.

Environment Setup

Installation via Image

For MindIE images, refer to Installation via Image.

Installation on Containers or Physical Machines

• For details about the software packages and dependencies to be prepared, see Preparing Software Packages and Dependencies.
• For details about how to install software packages and dependencies, see Installing Software Packages and Dependencies.

Compilation and Installation

1. Install Python. MindIE-LLM supports Python 3.10 and Python 3.11.

   pip install --upgrade pip
   pip install wheel setuptools

2. Run the following commands to clone the source code repository:

   git clone https://gitcode.com/Ascend/MindIE-LLM.git
   cd MindIE-LLM

3. Run the following command to compile the third-party dependencies:

   bash build.sh 3rd

4. Set the environment variable. Obtain the Python site-packages path (do not hardcode the torch path) and configure the dynamic library search path.

   TORCH_PATH=$(python3 -c "import torch, os; print(os.path.dirname(torch.__file__))")
   TORCH_NPU_PATH=$(python3 -c "import torch_npu, os; print(os.path.dirname(torch_npu.__file__))")
   export LD_LIBRARY_PATH=${TORCH_PATH}/lib:${TORCH_PATH}/../torch.libs:$LD_LIBRARY_PATH
   export PYTORCH_NPU_INSTALL_PATH=${TORCH_NPU_PATH}

   (Optional) Specify the version number of the generated .whl software package.

   export MINDIE_LLM_VERSION_OVERRIDE=3.0.0

5. Compile and generate the .whl software package of MindIE-LLM. Run the following command in the root directory of the source code:

   pip wheel . --no-build-isolation -v

   • After the compilation is complete, the mindie_llm-<version>-*.whl file is generated in the current directory.
   • During compilation, setup.py automatically invokes build.sh to compile C++ code and copies third-party dependencies to the package.
   • After the compilation, the temporary directory build, binary directory output, and debug symbol table llm_debug_symbols are generated.

6. Run the following command to install MindIE-LLM:

   old_umask=$(umask)
   umask 027
   pip install mindie_llm*.whl
   umask $old_umask

7. Compile the .whl software package of ATB_Models.

   cd examples/atb_models
   pip wheel . --no-build-isolation -v

   Note: If Python 3.10 is used for compilation, torch 2.9.0 and TorchNPU 2.9.0 must be used. Otherwise, the _bz2 module will be missing, causing compilation failure.

8. Run the following command to install ATB_Models:

   pip install atb_llm*.whl

9. (Optional) Configure the operating environment variables.

   After the installation is complete, set environment variables. Use Python to dynamically obtain the atb_llm installation path to adapt to different Python environments and site-packages locations.

   ATB_LLM_PATH=$(python3 -c "import atb_llm, os; print(os.path.dirname(atb_llm.__file__))")
   export ATB_SPEED_HOME_PATH=${ATB_LLM_PATH}
   export LD_LIBRARY_PATH=${ATB_LLM_PATH}/lib:${LD_LIBRARY_PATH}

   NOTE

   • You are advised to write the preceding commands into the ~/.bashrc or startup script to avoid manual setting each time.
   • This environment variable is built into the MindIE image. If you use the image, you do not need to manually set it.

Upgrade

For upgrade instructions, see Upgrade.

Uninstallation

For uninstallation instructions, see Uninstallation.