请按照表1所示,获取对应操作系统的软件包与打包镜像所需Dockerfile文件与脚本文件。
深度学习引擎软件包及MindSpore框架软件包名称中{version}表示版本号,{arch}表示架构。
软件包 |
说明 |
获取方法 |
---|---|---|
Ascend-cann-nnae_{version}_linux-{arch}.run |
深度学习引擎软件包。 |
|
mindspore_ascend-{version}-cp3x-cp3xm-linux_{arch}.whl |
MindSpore框架whl包。 当前可支持Python 3.7和Python 3.9,软件包名中x表示7或9,请根据实际情况选择对应软件包。
说明:
MindSpore 2.0.0版本后的软件包名由mindspore-ascend修改为mindspore。 |
|
Dockerfile |
制作镜像需要。 |
用户根据业务自行准备。 |
ascend_install.info |
驱动安装信息文件。 |
从host拷贝“/etc/ascend_install.info”文件。 |
version.info |
驱动版本信息文件。 |
从host拷贝“/usr/local/Ascend/driver/version.info”文件。 |
prebuild.sh |
执行训练运行环境安装准备工作,例如配置代理等。 |
用户根据业务自行准备。 |
install_ascend_pkgs.sh |
昇腾软件包安装脚本。 |
|
postbuild.sh |
清除不需要保留在容器中的安装包、脚本、代理配置等 |
为了防止软件包在传递过程或存储期间被恶意篡改,下载软件包时需下载对应的数字签名文件用于完整性验证。
在软件包下载之后,请参考《OpenPGP签名验证指南》,对从Support网站下载的软件包进行PGP数字签名校验。如果校验失败,请不要使用该软件包,先联系华为技术支持工程师解决。
使用软件包安装/升级之前,也需要按上述过程先验证软件包的数字签名,确保软件包未被篡改。
企业客户请访问:https://support.huawei.com/enterprise/zh/tool/pgp-verify-TL1000000054
本章节以Ubuntu操作系统为例。
vi prebuild.sh
vi install_ascend_pkgs.sh
vi postbuild.sh
vi Dockerfile
为获取镜像“ubuntu:18.04”,用户也可以通过执行docker pull ubuntu:18.04命令从Docker Hub拉取。
docker build -t 镜像名_系统架构:镜像tag .
docker build -t test_train_arm64:v1.0 .
命令解释如所表2示。
当出现“Successfully built xxx”表示镜像构建成功。
docker images
显示示例。
REPOSITORY TAG IMAGE ID CREATED SIZE test_train_arm64 v1.0 d82746acd7f0 27 minutes ago 749MB
使用过程中请根据实际情况修改软件包版本及架构。
#!/bin/bash
#--------------------------------------------------------------------------------
# 请在此处使用bash语法编写脚本代码,执行安装准备工作,例如配置代理等
# 本脚本将会在正式构建过程启动前被执行
#
# 注:本脚本运行结束后不会被自动清除,若无需保留在镜像中请在postbuild.sh脚本中清除
#--------------------------------------------------------------------------------
# dns设置
tee /etc/resolv.conf <<- EOF
nameserver xxx.xxx.xxx.xxx #DNS服务器IP,可填写多个,根据实际配置。
nameserver xxx.xxx.xxx.xxx
nameserver xxx.xxx.xxx.xxx
EOF
# apt代理设置
tee /etc/apt/apt.conf.d/80proxy <<- EOF
Acquire::http::Proxy "http://xxx.xxx.xxx.xxx:xxx"; #http代理服务器IP地址及端口。
Acquire::https::Proxy "http://xxx.xxx.xxx.xxx:xxx"; #https代理服务器IP地址及端口。
EOF
chmod 777 -R /tmp
rm /var/lib/apt/lists/*
#apt源设置(以ubuntu 18.04 Arm源为示例,请根据实际配置)
tee /etc/apt/sources.list <<- EOF
deb http://mirrors.aliyun.com/ubuntu-ports/ bionic main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu-ports/ bionic main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu-ports/ bionic-security main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu-ports/ bionic-security main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu-ports/ bionic-updates main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu-ports/ bionic-updates main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu-ports/ bionic-proposed main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu-ports/ bionic-proposed main restricted universe multiverse
deb http://mirrors.aliyun.com/ubuntu-ports/ bionic-backports main restricted universe multiverse
deb-src http://mirrors.aliyun.com/ubuntu-ports/ bionic-backports main restricted universe multiverse
EOF
#!/bin/bash
#--------------------------------------------------------------------------------
# 请在此处使用bash语法编写脚本代码,执行安装准备工作,例如配置代理等
# 本脚本将会在正式构建过程启动前被执行
#
# 注:本脚本运行结束后不会被自动清除,若无需保留在镜像中请在postbuild.sh脚本中清除
#--------------------------------------------------------------------------------
# apt代理设置
tee /etc/apt/apt.conf.d/80proxy <<- EOF
Acquire::http::Proxy "http://xxx.xxx.xxx.xxx:xxx"; #http代理服务器IP地址及端口
Acquire::https::Proxy "http://xxx.xxx.xxx.xxx:xxx"; #https代理服务器IP地址及端口
EOF
#apt源设置(以ubuntu 18.04 x86_64源为示例,请根据实际配置)
tee /etc/apt/sources.list <<- EOF
deb http://mirrors.ustc.edu.cn/ubuntu/ bionic main multiverse restricted universe
deb http://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main multiverse restricted universe
deb http://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main multiverse restricted universe
deb http://mirrors.ustc.edu.cn/ubuntu/ bionic-security main multiverse restricted universe
deb http://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main multiverse restricted universe
deb-src http://mirrors.ustc.edu.cn/ubuntu/ bionic main multiverse restricted universe
deb-src http://mirrors.ustc.edu.cn/ubuntu/ bionic-backports main multiverse restricted universe
deb-src http://mirrors.ustc.edu.cn/ubuntu/ bionic-proposed main multiverse restricted universe
deb-src http://mirrors.ustc.edu.cn/ubuntu/ bionic-security main multiverse restricted universe
deb-src http://mirrors.ustc.edu.cn/ubuntu/ bionic-updates main multiverse restricted universe
EOF
#!/bin/bash #-------------------------------------------------------------------------------- # 请在此处使用bash语法编写脚本代码,安装昇腾软件包 # # 注:本脚本运行结束后不会被自动清除,若无需保留在镜像中请在postbuild.sh脚本中清除 #-------------------------------------------------------------------------------- # 构建之前把host上的/etc/ascend_install.info拷贝一份到当前目录 cp ascend_install.info /etc/ mkdir -p /usr/local/Ascend/driver/ cp version.info /usr/local/Ascend/driver/ # Ascend-cann-nnae_{version}_linux-{arch}.run chmod +x Ascend-cann-nnae_{version}_linux-{arch}.run ./Ascend-cann-nnae_{version}_linux-{arch}.run --install-path=/usr/local/Ascend/ --install --quiet # 只安装nnae包,需要清理,容器启动时通过ascend docker挂载进来 rm -f version.info rm -rf /usr/local/Ascend/driver/
#!/bin/bash #-------------------------------------------------------------------------------- # 请在此处使用bash语法编写脚本代码,清除不需要保留在容器中的安装包、脚本、代理配置等 # 本脚本将会在正式构建过程结束后被执行 # # 注:本脚本运行结束后会被自动清除,不会残留在镜像中;脚本所在位置和Working Dir位置为/root #-------------------------------------------------------------------------------- rm -f ascend_install.info rm -f prebuild.sh rm -f install_ascend_pkgs.sh rm -f Dockerfile rm -f version.info rm -f Ascend-cann-nnae_{version}_linux-{arch}.run # 请根据实际安装的版本选择需要删除的包 rm -f mindspore_ascend-{version}-cp3x-cp3xm-linux_{arch}.whl rm -f /etc/apt/apt.conf.d/80proxy tee /etc/resolv.conf <<- EOF # This file is managed by man:systemd-resolved(8). Do not edit. # # This is a dynamic resolv.conf file for connecting local clients to the # internal DNS stub resolver of systemd-resolved. This file lists all # configured search domains. # # Run "systemd-resolve --status" to see details about the uplink DNS servers # currently in use. # # Third party programs must not access this file directly, but only through the # symlink at /etc/resolv.conf. To manage man:resolv.conf(5) in a different way, # replace this symlink by a static file or a different symlink. # # See man:systemd-resolved.service(8) for details about the supported modes of # operation for /etc/resolv.conf. options edns0 nameserver xxx.xxx.xxx.xxx nameserver xxx.xxx.xxx.xxx EOF
FROM ubuntu:18.04 ARG HOST_ASCEND_BASE=/usr/local/Ascend ARG INSTALL_ASCEND_PKGS_SH=install_ascend_pkgs.sh ARG NNAE_PATH=/usr/local/Ascend/nnae/latest ARG MINDSPORE_PKG=mindspore_ascend-{version}-cp39-cp39-linux_x86_64.whl ARG PREBUILD_SH=prebuild.sh ARG POSTBUILD_SH=postbuild.sh WORKDIR /tmp COPY . ./ # 触发prebuild.sh RUN bash -c "test -f $PREBUILD_SH && bash $PREBUILD_SH" ENV http_proxy http://xxx ENV https_proxy http://xxx # 系统包 RUN apt update && \ apt install --no-install-recommends curl g++ pkg-config unzip wget build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev \ libblas3 liblapack3 liblapack-dev openssl libssl-dev libblas-dev gfortran libhdf5-dev libffi-dev libicu60 libxml2 -y RUN wget https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz --no-check-certificate RUN tar -zxvf Python-3.9.2.tgz && cd Python-3.9.2 && ./configure --prefix=/usr/local/python3.9.2 --enable-shared && make && make install RUN ln -s /usr/local/python3.9.2/bin/python3.9 /usr/local/python3.9.2/bin/python && \ ln -s /usr/local/python3.9.2/bin/pip3.9 /usr/local/python3.9.2/bin/pip # 建立python软链接 #RUN ln -s /usr/bin/python3.9.2 /usr/bin/python # 配置Python pip源 RUN mkdir -p ~/.pip \ && echo '[global] \n\ index-url=https://pypi.doubanio.com/simple/\n\ trusted-host=pypi.doubanio.com' >> ~/.pip/pip.conf ENV LD_LIBRARY_PATH=/usr/local/python3.9.2/lib:$LD_LIBRARY_PATH ENV PATH=/usr/local/python3.9.2/bin:$PATH # 创建HwHiAiUser用户和属主,uid和gid请与物理机保持一致避免出现无属主文件。示例中会自动创建user和对应的group,uid和gid都为1000 RUN useradd -d /home/HwHiAiUser -u 1000 -m -s /bin/bash HwHiAiUser # Python包 RUN pip install numpy && \ pip install decorator && \ pip install sympy==1.4 && \ pip install cffi==1.12.3 && \ pip install pyyaml && \ pip install pathlib2 && \ pip install grpcio && \ pip install grpcio-tools && \ pip install protobuf && \ pip install scipy && \ pip install requests && \ pip install kubernetes && \ pip3.7 install attrs && \ pip3.7 install psutil && \ pip3.7 install absl-py # Ascend包 RUN umask 0022 && bash $INSTALL_ASCEND_PKGS_SH # MindSpore安装 RUN pip install $MINDSPORE_PKG ENV http_proxy "" ENV https_proxy "" # 触发postbuild.sh RUN bash -c "test -f $POSTBUILD_SH && bash $POSTBUILD_SH" && \ rm $POSTBUILD_SH
FROM ubuntu:18.04 ARG HOST_ASCEND_BASE=/usr/local/Ascend ARG INSTALL_ASCEND_PKGS_SH=install_ascend_pkgs.sh ARG NNAE_PATH=/usr/local/Ascend/nnae/latest ARG MINDSPORE_PKG=mindspore_ascend-{version}-cp39-cp39-linux_x86_64.whl ARG PREBUILD_SH=prebuild.sh ARG POSTBUILD_SH=postbuild.sh WORKDIR /tmp COPY . ./ # 触发prebuild.sh RUN bash -c "test -f $PREBUILD_SH && bash $PREBUILD_SH" ENV http_proxy http://xxx ENV https_proxy http://xxx # 系统包 RUN apt update && \ apt install --no-install-recommends curl g++ pkg-config unzip wget build-essential zlib1g-dev libncurses5-dev libgdbm-dev libnss3-dev libssl-dev libreadline-dev libffi-dev \ libblas3 liblapack3 liblapack-dev openssl libssl-dev libblas-dev gfortran libhdf5-dev libffi-dev libicu60 libxml2 -y RUN wget https://www.python.org/ftp/python/3.9.2/Python-3.9.2.tgz --no-check-certificate RUN tar -zxvf Python-3.9.2.tgz && cd Python-3.9.2 && ./configure --prefix=/usr/local/python3.9.2 --enable-shared && make && make install RUN ln -s /usr/local/python3.9.2/bin/python3.9 /usr/local/python3.9.2/bin/python && \ ln -s /usr/local/python3.9.2/bin/pip3.9 /usr/local/python3.9.2/bin/pip # 建立Python软链接 #RUN ln -s /usr/bin/python3.9.2 /usr/bin/python # 配置Python pip源 RUN mkdir -p ~/.pip \ && echo '[global] \n\ index-url=https://pypi.doubanio.com/simple/\n\ trusted-host=pypi.doubanio.com' >> ~/.pip/pip.conf ENV LD_LIBRARY_PATH=/usr/local/python3.9.2/lib:$LD_LIBRARY_PATH ENV PATH=/usr/local/python3.9.2/bin:$PATH # 创建HwHiAiUser用户和属主,uid和gid请与物理机保持一致避免出现无属主文件。示例中会自动创建user和对应的group,uid和gid都为1000 RUN useradd -d /home/HwHiAiUser -u 1000 -m -s /bin/bash HwHiAiUser # Python包 RUN pip install numpy && \ pip install decorator && \ pip install sympy==1.4 && \ pip install cffi==1.12.3 && \ pip install pyyaml && \ pip install pathlib2 && \ pip install grpcio && \ pip install grpcio-tools && \ pip install protobuf && \ pip install scipy && \ pip install requests && \ pip install kubernetes && \ pip3.7 install attrs && \ pip3.7 install psutil && \ pip3.7 install absl-py # Ascend包 RUN umask 0022 && bash $INSTALL_ASCEND_PKGS_SH # MindSpore安装 RUN pip install $MINDSPORE_PKG ENV http_proxy "" ENV https_proxy "" # 触发postbuild.sh RUN bash -c "test -f $POSTBUILD_SH && bash $POSTBUILD_SH" && \ rm $POSTBUILD_SH
FROM ubuntu:18.04 ARG HOST_ASCEND_BASE=/usr/local/Ascend ARG INSTALL_ASCEND_PKGS_SH=install_ascend_pkgs.sh ARG NNAE_PATH=/usr/local/Ascend/nnae/latest ARG MINDSPORE_PKG=mindspore_ascend-{version}-cp37-cp37m-linux_aarch64.whl ARG PREBUILD_SH=prebuild.sh ARG POSTBUILD_SH=postbuild.sh WORKDIR /tmp COPY . ./ # 触发prebuild.sh RUN bash -c "test -f $PREBUILD_SH && bash $PREBUILD_SH" ENV http_proxy http://xxx.xxx.xxx.xxx:xxx ENV https_proxy http://xxx.xxx.xxx.xxx:xxx # 系统包 RUN apt update && \ apt install --no-install-recommends python3.7 python3.7-dev curl g++ pkg-config unzip \ libblas3 liblapack3 liblapack-dev libblas-dev gfortran libhdf5-dev libffi-dev libicu60 libxml2 \ libgmp-dev -y # 建立Python软链接 RUN ln -s /usr/bin/python3.7 /usr/bin/python # 配置Python pip源 RUN mkdir -p ~/.pip \ && echo '[global] \n\ index-url=https://pypi.doubanio.com/simple/\n\ trusted-host=pypi.doubanio.com' >> ~/.pip/pip.conf # pip3.7 RUN curl -k https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \ cd /tmp && \ apt-get download python3-distutils && \ dpkg-deb -x python3-distutils_*.deb / && \ rm python3-distutils_*.deb && \ cd - && \ python3.7 get-pip.py && \ rm get-pip.py # 创建HwHiAiUser用户和属主,uid和gid请与物理机保持一致避免出现无属主文件。示例中会自动创建user和对应的group,uid和gid都为1000 RUN useradd -d /home/HwHiAiUser -u 1000 -m -s /bin/bash HwHiAiUser # Python包 RUN pip3.7 install numpy && \ pip3.7 install decorator && \ pip3.7 install sympy==1.4 && \ pip3.7 install cffi==1.12.3 && \ pip3.7 install pyyaml && \ pip3.7 install pathlib2 && \ pip3.7 install grpcio && \ pip3.7 install grpcio-tools && \ pip3.7 install protobuf && \ pip3.7 install scipy && \ pip3.7 install requests && \ pip3.7 install easydict && \ pip3.7 install attrs && \ pip3.7 install psutil && \ pip3.7 install absl-py # Ascend包 RUN umask 0022 && bash $INSTALL_ASCEND_PKGS_SH # MindSpore安装 RUN pip3.7 install $MINDSPORE_PKG # 创建/lib64/ld-linux-aarch64.so.1 RUN umask 0022 && \ if [ ! -d "/lib64" ]; \ then \ mkdir /lib64 && ln -sf /lib/ld-linux-aarch64.so.1 /lib64/ld-linux-aarch64.so.1; \ fi ENV http_proxy "" ENV https_proxy "" # 触发postbuild.sh RUN bash -c "test -f $POSTBUILD_SH && bash $POSTBUILD_SH" && \ rm $POSTBUILD_SH
FROM ubuntu:18.04 ARG HOST_ASCEND_BASE=/usr/local/Ascend ARG INSTALL_ASCEND_PKGS_SH=install_ascend_pkgs.sh ARG NNAE_PATH=/usr/local/Ascend/nnae/latest ARG MINDSPORE_PKG=mindspore_ascend-{version}-cp37-cp37m-linux_x86_64.whl ARG PREBUILD_SH=prebuild.sh ARG POSTBUILD_SH=postbuild.sh WORKDIR /tmp COPY . ./ # 触发prebuild.sh RUN bash -c "test -f $PREBUILD_SH && bash $PREBUILD_SH" ENV http_proxy http://xxx.xxx.xxx.xxx:xxx ENV https_proxy http://xxx.xxx.xxx.xxx:xxx # 系统包 RUN apt update && \ apt install --no-install-recommends python3.7 python3.7-dev curl g++ pkg-config unzip \ libblas3 liblapack3 liblapack-dev libblas-dev gfortran libhdf5-dev libffi-dev libicu60 libxml2 \ libgmp-dev -y # 建立Python软链接 RUN ln -s /usr/bin/python3.7 /usr/bin/python # 配置Python pip源 RUN mkdir -p ~/.pip \ && echo '[global] \n\ index-url=https://pypi.doubanio.com/simple/\n\ trusted-host=pypi.doubanio.com' >> ~/.pip/pip.conf # pip3.7 RUN curl -k https://bootstrap.pypa.io/get-pip.py -o get-pip.py && \ cd /tmp && \ apt-get download python3-distutils && \ dpkg-deb -x python3-distutils_*.deb / && \ rm python3-distutils_*.deb && \ cd - && \ python3.7 get-pip.py && \ rm get-pip.py # 创建HwHiAiUser用户和属主,uid和gid请与物理机保持一致避免出现无属主文件。示例中会自动创建user和对应的group,uid和gid都为1000 RUN useradd -d /home/HwHiAiUser -u 1000 -m -s /bin/bash HwHiAiUser # Python包 RUN pip3.7 install numpy && \ pip3.7 install decorator && \ pip3.7 install sympy==1.4 && \ pip3.7 install cffi==1.12.3 && \ pip3.7 install pyyaml && \ pip3.7 install pathlib2 && \ pip3.7 install grpcio && \ pip3.7 install grpcio-tools && \ pip3.7 install protobuf && \ pip3.7 install scipy && \ pip3.7 install requests && \ pip3.7 install easydict && \ pip3.7 install attrs && \ pip3.7 install psutil && \ pip3.7 install absl-py # Ascend包 RUN umask 0022 && bash $INSTALL_ASCEND_PKGS_SH # MindSpore安装 RUN pip3.7 install $MINDSPORE_PKG ENV http_proxy "" ENV https_proxy "" # 触发postbuild.sh RUN bash -c "test -f $POSTBUILD_SH && bash $POSTBUILD_SH" && \ rm $POSTBUILD_SH