本章节指导用户如何制作MindIE镜像包,仅用于MindIE MS部署服务时使用。
MindIE镜像包仅用于MindIE MS部署服务时使用,其他安装场景请使用容器化部署MindIE章节。
硬件 |
操作系统(建议) |
---|---|
Arm: Ubuntu 22.04 |
|
Atlas 300I Duo 推理卡+Atlas 800 推理服务器(型号:3000) |
Arm: Ubuntu 20.04 |
制作MindIE镜像包对所依赖的软件版本有特定的要求,具体所需版本请参见表1。
软件类型 |
软件包名称 |
软件说明 |
获取链接 |
---|---|---|---|
MindIE |
Ascend-mindie_1.0.0_linux-aarch64.run |
推理引擎软件包。 |
|
CANN |
Ascend-cann-toolkit_8.0.0_linux-aarch64.run |
开发套件包。 |
|
说明:
以上软件包名中的910x和310x请根据具体的硬件型号进行替换。 |
二进制算子包。 |
||
Ascend-cann-nnal_8.0.0_linux-aarch64.run |
加速库软件包 |
||
ATB Models |
Ascend-mindie-atb-models_1.0.0_linux-aarch64_py310_torch2.1.0-abi0.tar.gz |
模型库安装包。 |
|
Ascend Extension for PyTorch |
torch_npu-2.1.0.post10-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
torch_npu插件whl包。 |
说明:
在Pytorch栏单击对应版本后方下载按钮,跳转至Pytorch的gitee仓发布页,然后在页面下方获取对应版本的torch_npu。 |
torch-2.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl |
PyTorch框架2.1.0版本的whl包。 |
apt update
如果命令执行报错或者后续安装依赖时等待时间过长甚至报错,则检查网络是否连接或者把“/etc/apt/sources.list”文件中的源更换为可用的源或使用镜像源(以配置华为镜像源为例,可参考华为开源镜像站)。
以
制作镜像时非root用户ID应该和执行环境上安装的驱动用户ID保持一致,或安装驱动时使用--install-for-all参数为所有用户安装。
docker pull ubuntu:22.04
docker images | grep ubuntu
屏幕显示以下类似结果,则说明镜像已拉取成功:
ubuntu 22.04 981912c48e9a 7 weeks ago 69.2MB
├── Ascend-cann-kernels-910b_8.0.0_linux-aarch64.run ├── Ascend-cann-nnal_8.0.0_linux-aarch64.run ├── Ascend-cann-toolkit_8.0.0_linux-aarch64.run ├── Ascend-mindie_1.0.0_linux-aarch64.run ├── Ascend-mindie-atb-models_1.0.0_linux-aarch64_torch2.1.0-abi0.tar.gz ├── docker │ ├── docker_build.sh │ └── Dockerfile ├── install_cann.sh ├── install_mindie.sh ├── install_pta.sh ├── requirements-2.1.0.txt ├── server.js ├── torch_npu-2.1.0.post10-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl └── torch-2.1.0-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl
其中加粗文件为用户自行编写的文件。
docker build \ --build-arg no_proxy=127.0.0.1,localhost,local,.local,172.17.0.1 \ --build-arg DEVICE=910b \ --build-arg ARCH=aarch64 \ --build-arg CANN_VERSION=8.0.0 \ --build-arg TORCH_VERSION=2.1.0 \ --build-arg MINDIE_VERSION=1.0.0 \ --build-arg PY_VERSION=310 \ -t mindie:1.0.0-aarch64-800I-A2 \ --target mindie .
# Please make sure all `ARG` have been set correctly # Set the arguments for different images FROM ubuntu:22.04 AS base ARG UBUNTU_VERSION=22.04 ARG ARCH ARG DEVICE LABEL description="Image for ${DEVICE} based on Ubuntu${UBUNTU_VERSION} ${ARCH}" RUN groupadd -g 1000 mindie && \ useradd -u 1000 -g 1000 -m mindieuser ENV PYTHONPATH=/home/mindieuser/.local/lib/python3.10/site-packages:$PATH ENV PATH=/home/mindieuser/.local/bin:$PYTHONPATH RUN echo 'export LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64/driver:$LD_LIBRARY_PATH' >> /home/mindieuser/.bashrc && \ echo 'export LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64/common:$LD_LIBRARY_PATH' >> /home/mindieuser/.bashrc && \ echo 'export PYTHONPATH=/home/mindieuser/.local/lib/python3.10/site-packages:$PATH' >> /home/mindieuser/.bashrc && \ echo 'export PATH=/home/mindieuser/.local/bin:$PYTHONPATH' >> /home/mindieuser/.bashrc && \ echo "export LANG=en_US.UTF-8" >> /home/mindieuser/.bashrc && \ echo "export LANGUAGE=en_US:en" >> /home/mindieuser/.bashrc && \ echo "export LC_ALL=en_US.UTF-8" >> /home/mindieuser/.bashrc && \ # Configure the Ubuntu mirror. An example is provided below: apt-get update && \ apt-get install --no-install-recommends -y ca-certificates && \ DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y tzdata && \ apt-get install --no-install-recommends -y gcc g++ make cmake && \ apt-get install --no-install-recommends -y zlib1g zlib1g-dev libbz2-dev liblzma-dev libffi-dev libssl-dev openssl libsqlite3-dev && \ apt-get install --no-install-recommends -y libblas-dev liblapack-dev libopenblas-dev libblas3 liblapack3 gfortran libhdf5-dev && \ apt-get install --no-install-recommends -y wget curl pkg-config vim libxml2 libxslt1-dev locales && \ apt-get install --no-install-recommends -y pciutils net-tools ipmitool numactl linux-tools-common && \ apt-get install --no-install-recommends -y libgl1-mesa-glx libgmpxx4ldbl && \ apt-get install --no-install-recommends -y xz-utils unzip && \ apt-get install --no-install-recommends -y python3-pip python-is-python3 && \ # Configure the pip mirror. An example is provided below: # pip config --user set global.index https://mirrors.tools.huawei.com/pypi && \ # pip config --user set global.index-url https://mirrors.tools.huawei.com/pypi/simple && \ # pip config --user set global.trusted-host mirrors.tools.huawei.com && \ locale-gen en_US.UTF-8 WORKDIR /home/mindieuser USER mindieuser ENV TZ=Asia/Shanghai ####################################################################################### # docker build -t cann --target cann . ####################################################################################### FROM base AS cann ARG DEVICE ARG ARCH ARG CANN_VERSION RUN echo "source /home/mindieuser/Ascend/ascend-toolkit/set_env.sh" >> /home/mindieuser/.bashrc && \ echo "source /home/mindieuser/Ascend/nnal/atb/set_env.sh" >> /home/mindieuser/.bashrc && \ wget -q http://172.17.0.1:3000/Ascend-cann-toolkit_${CANN_VERSION}_linux-${ARCH}.run -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/Ascend-cann-kernels-${DEVICE}_${CANN_VERSION}_linux-${ARCH}.run -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/Ascend-cann-nnal_${CANN_VERSION}_linux-${ARCH}.run -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/install_cann.sh -P /home/mindieuser/package/ && \ cd /home/mindieuser/package && \ bash install_cann.sh && \ rm -rf /home/mindieuser/package/* ####################################################################################### # docker build -t pta --target pta . ####################################################################################### FROM cann AS pta ARG DEVICE ARG ARCH ARG TORCH_VERSION RUN wget -q http://172.17.0.1:3000/torch_npu-${TORCH_VERSION}.post10-cp310-cp310-manylinux_2_17_${ARCH}.manylinux2014_${ARCH}.whl -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/torch-${TORCH_VERSION}-cp310-cp310-manylinux_2_17_${ARCH}.manylinux2014_${ARCH}.whl -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/requirements-${TORCH_VERSION}.txt -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/install_pta.sh -P /home/mindieuser/package/ && \ cd /home/mindieuser/package && \ pip install -r requirements-${TORCH_VERSION}.txt --no-cache-dir && \ bash install_pta.sh && \ pip cache purge && \ rm -rf /home/mindieuser/package/* ####################################################################################### # docker build -t mindie --target mindie . ####################################################################################### FROM pta AS mindie ARG DEVICE ARG ARCH ARG TORCH_VERSION ARG MINDIE_VERSION RUN echo "source /home/mindieuser/Ascend/mindie/set_env.sh" >> /home/mindieuser/.bashrc && \ echo "source /home/mindieuser/Ascend/llm_model/set_env.sh" >> /home/mindieuser/.bashrc && \ wget -q http://172.17.0.1:3000/Ascend-mindie_${MINDIE_VERSION}_linux-${ARCH}.run -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/Ascend-mindie-atb-models_${MINDIE_VERSION}_linux-${ARCH}_py310_torch${TORCH_VERSION}-abi0.tar.gz -P /home/mindieuser/package/ && \ wget -q http://172.17.0.1:3000/install_mindie.sh -P /home/mindieuser/package/ && \ cd /home/mindieuser/package && \ bash install_mindie.sh && \ pip cache purge && \ rm -rf /home/mindieuser/package/*
https://www.python.org/ftp/python/3.11.4/Python-3.11.4.tgz
FROM ubuntu:22.04 AS base ARG UBUNTU_VERSION=22.04 ARG PYTHON_VERSION=3.11.4 # Set python version ARG ARCH ARG DEVICE LABEL description="Image for ${DEVICE} based on Ubuntu${UBUNTU_VERSION} ${ARCH}" ENV LD_LIBRARY_PATH="/usr/local/lib:$LD_LIBRARY_PATH" RUN echo 'export LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64/driver:$LD_LIBRARY_PATH' >> /home/mindieuser/.bashrc && \ echo 'export LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64/common:$LD_LIBRARY_PATH' >> /home/mindieuser/.bashrc && \ echo 'export PYTHONPATH=/home/mindieuser/.local/lib/python3.11.4/site-packages:$PATH' >> /home/mindieuser/.bashrc && \ echo 'export PATH=/home/mindieuser/.local/bin:$PYTHONPATH' >> /home/mindieuser/.bashrc && \ echo "export LANG=en_US.UTF-8" >> /home/mindieuser/.bashrc && \ echo "export LANGUAGE=en_US:en" >> /home/mindieuser/.bashrc && \ echo "export LC_ALL=en_US.UTF-8" >> /home/mindieuser/.bashrc && \ # Configure the Ubuntu mirror. An example is provided below: apt-get update && \ apt-get install --no-install-recommends -y ca-certificates && \ DEBIAN_FRONTEND=noninteractive apt-get install --no-install-recommends -y tzdata && \ apt-get install --no-install-recommends -y gcc g++ make cmake && \ apt-get install --no-install-recommends -y zlib1g zlib1g-dev libbz2-dev liblzma-dev libffi-dev libssl-dev openssl libsqlite3-dev && \ apt-get install --no-install-recommends -y libblas-dev liblapack-dev libopenblas-dev libblas3 liblapack3 gfortran libhdf5-dev && \ apt-get install --no-install-recommends -y wget curl pkg-config vim libxml2 libxslt1-dev locales && \ apt-get install --no-install-recommends -y pciutils net-tools ipmitool numactl linux-tools-common && \ apt-get install --no-install-recommends -y libgl1-mesa-glx libgmpxx4ldbl && \ apt-get install --no-install-recommends -y xz-utils unzip && \ wget -q http://172.17.0.1:3000/Python-$PYTHON_VERSION.tgz -P /opt/package/ && \ cd /opt/package && \ tar -xf Python-$PYTHON_VERSION.tgz && \ cd Python-$PYTHON_VERSION && \ ./configure --enable-optimizations --enable-shared --enable-loadable-sqlite-extensions --with-lto --with-ensurepip --with-computed-gotos && \ make -j$(nproc) && \ make altinstall && \ cd .. && \ rm -rf Python-$PYTHON_VERSION Python-$PYTHON_VERSION.tgz && \ ln -sf /usr/local/bin/python3.11 /usr/bin/python3 && \ ln -sf /usr/local/bin/python3.11 /usr/bin/python && \ python -m ensurepip --default-pip && \ ln -sf /usr/local/bin/pip3.11 /usr/bin/pip3 && \ ln -sf /usr/local/bin/pip3.11 /usr/bin/pip && \ # Configure the pip mirror. An example is provided below: # pip config --user set global.index https://mirrors.tools.huawei.com/pypi && \ # pip config --user set global.index-url https://mirrors.tools.huawei.com/pypi/simple && \ # pip config --user set global.trusted-host mirrors.tools.huawei.com && \ locale-gen en_US.UTF-8 ENV TZ=Asia/Shanghai
#!/bin/bash CANN_TOOKIT="Ascend-cann-toolkit_*_linux-*.run" CANN_KERNELS="Ascend-cann-kernels-*_*_linux-*.run" CANN_NNAL="Ascend-cann-nnal_*_linux-*.run" chmod +x *.run yes | ./${CANN_TOOKIT} --install --quiet toolkit_status=$? if [ ${toolkit_status} -eq 0 ]; then echo "install toolkit successfully" else echo "install toolkit failed with status ${toolkit_status}" fi yes | ./${CANN_KERNELS} --install --quiet kernels_status=$? if [ ${kernels_status} -eq 0 ]; then echo "install kernels successfully" else echo "install kernels failed with status ${kernels_status}" fi source /home/mindieuser/Ascend/ascend-toolkit/set_env.sh yes | ./${CANN_NNAL} --install --quiet nnal_status=$? if [ ${nnal_status} -eq 0 ]; then echo "install nnal successfully" else echo "install nnal failed with status ${nnal_status}" fi
#!/bin/bash source /home/mindieuser/Ascend/ascend-toolkit/set_env.sh mkdir -p /home/mindieuser/Ascend/llm_model MINDIE="Ascend-mindie_*_linux-*.run" MODEL="Ascend-mindie-atb-models_*_linux-aarch64_py310_torch2.1.0-abi0.tar.gz" chmod +x *.run tar -xzf ./${MODEL} -C /home/mindieuser/Ascend/llm_model yes | ./${MINDIE} --install --quiet 2> /dev/null mindie_status=$? if [ ${mindie_status} -eq 0 ]; then echo "install mindie successfully" else echo "install mindie failed with status ${mindie_status}" fi
#!/bin/bash pip3 install torch-2.1.0-cp310-cp310-manylinux_2_17_*.manylinux2014_*.whl TORCH_NPU_IN_PYTORCH_MANYLINUX=torch_npu-2.1.0*aarch64.whl echo "start install torch_npu, wait for a minute..." pip install ${TORCH_NPU_IN_PYTORCH_MANYLINUX}
# # This file is autogenerated by pip-compile with Python 3.10 # by the following command: # # pip-compile absl-py==2.1.0 # via rouge-score accelerate==0.34.2 # via -r requirements.in annotated-types==0.7.0 # via pydantic attrs==24.2.0 # via jsonlines brotli==1.1.0 # via geventhttpclient certifi==2024.8.30 # via # geventhttpclient # requests cffi==1.17.1 # via -r requirements.in charset-normalizer==3.3.2 # via requests click==8.1.7 # via nltk cloudpickle==3.0.0 # via -r requirements.in colorama==0.4.6 # via sacrebleu contourpy==1.3.0 # via matplotlib cpm-kernels==1.0.11 # via -r requirements.in cycler==0.12.1 # via matplotlib decorator==5.1.1 # via -r requirements.in et-xmlfile==1.1.0 # via openpyxl filelock==3.16.1 # via # huggingface-hub # icetk # torch # transformers fonttools==4.54.1 # via matplotlib fsspec==2024.9.0 # via # huggingface-hub # torch fuzzywuzzy==0.18.0 # via -r requirements.in gevent==24.2.1 # via geventhttpclient geventhttpclient==2.3.1 # via -r requirements.in greenlet==3.1.1 # via gevent grpcio==1.66.1 # via tritonclient huggingface-hub==0.25.1 # via # accelerate # tokenizers # transformers icetk==0.0.4 # via -r requirements.in idna==3.10 # via requests jieba==0.42.1 # via -r requirements.in jinja2==3.1.4 # via torch joblib==1.4.2 # via nltk jsonlines==4.0.0 # via -r requirements.in kiwisolver==1.4.7 # via matplotlib latex2mathml==3.77.0 # via mdtex2html loguru==0.7.2 # via -r requirements.in lxml==5.3.0 # via sacrebleu markdown==3.7 # via mdtex2html markupsafe==2.1.5 # via jinja2 matplotlib==3.9.2 # via -r requirements.in mdtex2html==1.3.0 # via -r requirements.in ml-dtypes==0.5.0 # via -r requirements.in mpmath==1.3.0 # via sympy networkx==3.3 # via torch nltk==3.9.1 # via rouge-score numpy==1.26.4 # via # -r requirements.in # accelerate # contourpy # matplotlib # ml-dtypes # pandas # pyarrow # rouge-score # sacrebleu # scipy # torchvision # transformers # tritonclient openpyxl==3.1.5 # via -r requirements.in packaging==24.1 # via # accelerate # huggingface-hub # matplotlib # transformers # tritonclient pandas==2.2.3 # via -r requirements.in pathlib2==2.3.7.post1 # via -r requirements.in pillow==10.4.0 # via # matplotlib # torchvision portalocker==2.10.1 # via sacrebleu prettytable==3.11.0 # via -r requirements.in protobuf==3.20.0 # via # -r requirements.in # tritonclient psutil==6.0.0 # via accelerate pyarrow==17.0.0 # via -r requirements.in pycparser==2.22 # via cffi pydantic==2.9.2 # via -r requirements.in pydantic-core==2.23.4 # via pydantic pyparsing==3.1.4 # via matplotlib python-dateutil==2.9.0.post0 # via # matplotlib # pandas python-rapidjson==1.20 # via tritonclient pytz==2024.2 # via pandas pyyaml==6.0.2 # via # accelerate # huggingface-hub # transformers rapidfuzz==3.10.0 # via thefuzz regex==2024.9.11 # via # nltk # sacrebleu # tiktoken # transformers requests==2.32.3 # via # huggingface-hub # icetk # tiktoken # torchvision # transformers rouge==1.0.1 # via -r requirements.in rouge-score==0.1.2 # via -r requirements.in sacrebleu==2.4.3 # via -r requirements.in safetensors==0.4.5 # via # accelerate # transformers scipy==1.14.1 # via -r requirements.in sentencepiece==0.2.0 # via icetk six==1.16.0 # via # pathlib2 # python-dateutil # rouge # rouge-score sympy==1.13.3 # via torch tabulate==0.9.0 # via sacrebleu termcolor==2.4.0 # via -r requirements.in thefuzz==0.22.1 # via -r requirements.in tiktoken==0.7.0 # via -r requirements.in tokenizers==0.19.1 # via transformers torch==2.1.0 # via # -r requirements.in # accelerate # torchvision torchvision==0.16.0 # via icetk tornado==6.4.1 # via -r requirements.in tqdm==4.66.5 # via # huggingface-hub # icetk # nltk # transformers transformers==4.44.0 # via -r requirements.in tritonclient[grpc]==2.49.0 # via -r requirements.in typing-extensions==4.12.2 # via # huggingface-hub # pydantic # pydantic-core # torch tzdata==2024.2 # via pandas urllib3==2.2.3 # via # geventhttpclient # requests # tritonclient wcwidth==0.2.13 # via prettytable wheel==0.44.0 # via -r requirements.in zope-event==5.0 # via gevent zope-interface==7.0.3 # via gevent # The following packages are considered to be unsafe in a requirements file: # setuptools
const http = require('http'); const fs = require('fs'); const path = require('path'); const port = 3000; const directory = __dirname; const server = http.createServer((req, res) => { const filePath = path.join(directory, req.url); if (req.url ==='/files') { // return all file names in current directory fs.readdir(directory, (err, files)=> { if (err) { res.writeHead(500, {'Content-Type': 'text/plain'}); res.end('Internal Server Error\n'); return; } res.writeHead(200, {'Content-Type': 'application/json'}); res.end(JSON.stringify(files)); }); }else { fs.stat(filePath, (err, stats) => { if (err || !stats.isFile()) { res.writeHead(404, { 'Content-Type': 'text/plain' }); res.end('Not Found\n'); return; } fs.createReadStream(filePath).pipe(res); }); } }); server.listen(port, () => { console.log(`Server is running at http://localhost:${port}`); });
wget -q http://172.17.0.1:3000/Ascend-cann-toolkit_${CANN_VERSION}_linux-${ARCH}.run -P /opt/package/
cd /home/package # 进入构建目录 node server.js
Server is running at http://localhost:3000
此时请保留该命令行窗口,等待构建完成后再行关闭。
cd /home/package/docker # 进入docker目录 bash docker_build.sh
当镜像构建完成时,会出现类似如下显示:
[+] Building 798.2s (9/9) FINISHED docker:default => [internal] load .dockerignore 0.0s => => transferring context: 2B 0.0s => [internal] load build definition from Dockerfile 0.0s => => transferring dockerfile: 6.12kB 0.0s => [internal] load metadata for docker.io/library/ubuntu:22.04 0.0s => CACHED [base 1/2] FROM docker.io/library/ubuntu:22.04 0.0s => [base 2/2] RUN echo 'export LD_LIBRARY_PATH=/usr/local/Ascend/driver/lib64/driver:$LD_LIBRARY_PATH' >> /root/.bashrc && echo 'export LD_LIBRARY_PATH=/usr/local/Ascend/drive 187.2s => [cann 1/1] RUN echo "source /usr/local/Ascend/ascend-toolkit/set_env.sh" >> /home/mindieuser/.bashrc && echo "source /usr/local/Ascend/nnal/atb/set_env.sh" >> /home/mindieuser/.bashrc && wget 274.4s => [pta 1/1] RUN wget -q http://172.17.0.1:3000/torch_npu-2.1.0.post8-cp310-cp310-manylinux_2_17_aarch64.manylinux2014_aarch64.whl -P /opt/package/ && wget -q http://172.17.0. 272.9s => [mindie 1/1] RUN echo "source /usr/local/Ascend/mindie/set_env.sh" >> /home/mindieuser/.bashrc && echo "source /usr/local/Ascend/llm_model/set_env.sh" >> /home/mindieuser/.bashrc && wget -q 31.6s => exporting to image 31.9s => => exporting layers 31.9s => => writing image sha256:ddc1229a39be3e2b9f2d0d88e809a4dc2db17ac9ec67c4c178c21fe1359eb6d7 0.0s => => naming to docker.io/library/mindie:1.0.0-aarch64-800I-A2 0.0s
然后执行以下命令:
docker images
当回显以下信息,则表示镜像构建成功:
1 2 | REPOSITORY TAG IMAGE ID CREATED SIZE mindie 1.0.0-aarch64-800I-A2 ddc1229a39be About a minute ago 12.3GB |