制作MindFormers训练镜像（MindSpore框架）

准备软件包

请按照表1所示，获取对应操作系统的软件包，并准备镜像所需的Dockerfile文件与脚本文件。

操作步骤

1. 在宿主机上完成软件包的准备工作。
2. 构建如下的Dockerfile。

   FROM ubuntu:20.04
    
   WORKDIR /root
    
   COPY . .
    
   ARG HOST_ASCEND_BASE=/usr/local/Ascend
   ARG TOOLKIT_PATH=/usr/local/Ascend/toolkit/latest
   ARG TOOLKIT=Ascend-cann-toolkit_8.1.RC1_linux-aarch64.run
   ARG KERNEL=Ascend-cann-kernels-910b_8.1.RC1_linux-aarch64.run
   ARG MINDX_ELASTIC_WHL=mindx_elastic-7.0.RC1-py3-none-linux_aarch64.whl
   ARG MINDIO_TTP_WHL=mindio_ttp-1.0.0-py3-none-linux_aarch64.whl
   ARG MINDFORMERS=mindformers
   ARG MINDSPORE_REQUIREMENTS=requirements.txt
   ARG MINDSPORE_WHL=mindspore-2.5.0-cp310-cp310-linux_aarch64.whl
   ARG TASKD_WHL=taskd-7.0.RC1-py3-none-linux_aarch64.whl    
    
   RUN echo "nameserver 114.114.114.114" > /etc/resolv.conf
    
   RUN echo "deb http://repo.huaweicloud.com/ubuntu-ports/ focal main restricted universe multiverse\n\
   deb http://repo.huaweicloud.com/ubuntu-ports/ focal-updates main restricted universe multiverse\n\
   deb http://repo.huaweicloud.com/ubuntu-ports/ focal-backports main restricted universe multiverse\n\
   deb http://ports.ubuntu.com/ubuntu-ports/ focal-security main restricted universe multiverse" > /etc/apt/sources.list
    
    
   ARG DEBIAN_FRONTEND=noninteractive
    
   RUN umask 0022 && apt update && \
       apt-get install -y --no-install-recommends \
       software-properties-common
   RUN umask 0022 && add-apt-repository ppa:deadsnakes/ppa && \
       apt update && \
       apt autoremove -y python python3 && \
       apt install -y python3.10 python3.10-dev
    
   # 建立Python软链接
   RUN ln -s /usr/bin/python3.10 /usr/bin/python
   RUN ln -s /usr/bin/python3.10 /usr/bin/python3
   RUN ln -s /usr/bin/python3.10-config /usr/bin/python-config
   RUN ln -s /usr/bin/python3.10-config /usr/bin/python3-config
    
   # 系统包
   RUN umask 0022 && apt update && \
       apt-get install -y --no-install-recommends \
           gcc g++ make cmake vim \
           zlib1g zlib1g-dev \
           openssl libsqlite3-dev libssl-dev \
           libffi-dev unzip pciutils \
           net-tools libblas-dev \
           gfortran libblas3 libopenblas-dev \
           curl unzip liblapack3 liblapack-dev \
           libhdf5-dev libxml2 patch
    
   # 时区
   # RUN ln -sf /usr/share/zoneinfo/Asia/Shanghai /etc/localtime
   RUN ln -sf /usr/share/zoneinfo/UTC /etc/localtime
    
   # 配置pip源
   RUN mkdir -p ~/.pip \
   && echo '[global] \n\
   index-url=https://mirrors.huaweicloud.com/repository/pypi/simple\n\
   trusted-host=mirrors.huaweicloud.com' >> ~/.pip/pip.conf
    
   # pip3.10
   RUN cd /tmp && \
       apt-get download python3-distutils && \
       dpkg-deb -x python3-distutils_*.deb / && \
       rm python3-distutils_*.deb && \
       cd - && \
       python get-pip.py && \
   rm get-pip.py
    
   RUN umask 0022 && \
       pip install sympy==1.4 && \
       pip install cffi && \
       pip install pathlib2 && \
       pip install grpcio && \
       pip install grpcio-tools && \
       pip install absl-py && \
       pip install datasets && \
       pip install tokenizers==0.20.1 && \
       pip install pyOpenSSL
    
   # 创建HwHiAiUser用户和属主，UID和GID请与物理机保持一致避免出现无属主文件。示例中会自动创建user和对应的group，UID和GID都为1000
   RUN useradd -d /home/HwHiAiUser -u 1000 -m -s /bin/bash HwHiAiUser
    
   # Ascend包
   # 构建之前把host的/usr/local/Ascend/driver/version.info拷贝一份到当前目录
   RUN umask 0022 &&  \
       cp ascend_install.info /etc/ && \
       mkdir -p /usr/local/Ascend/driver/ && \
       cp version.info /usr/local/Ascend/driver/ && \
       chmod +x $TOOLKIT && \
       chmod +x $KERNEL
    
   RUN umask 0022 && ./$TOOLKIT --install-path=/usr/local/Ascend/ --install --quiet
   RUN echo "source /usr/local/Ascend/ascend-toolkit/set_env.sh" >> ~/.bashrc
   RUN umask 0022 && ./$KERNEL --install --quiet
    
   # 只为了安装toolkit包，所以需要清理，容器启动时通过ascend docker挂载进来
   RUN rm -f version.info && \
       rm -rf /usr/local/Ascend/driver/
    
   # 安装mindspore
   RUN umask 0022 && pip uninstall te topi hccl -y && \
            pip install sympy && \
            pip install /usr/local/Ascend/ascend-toolkit/latest/lib64/te-*-py3-none-any.whl && \
            pip install /usr/local/Ascend/ascend-toolkit/latest/lib64/hccl-*-py3-none-any.whl
   RUN umask 0022 && \
       pip install -r $MINDSPORE_REQUIREMENTS && \
       pip install $MINDSPORE_WHL
    
   # 安装mindformers
   RUN umask 0022 && cd $MINDFORMERS && \
       pip install -r requirements.txt && bash build.sh
    
   # MindCluster无损失断点续训适配脚本
   RUN umask 0022 && \
       pip install $MINDX_ELASTIC_WHL && \
       pip install $MINDIO_TTP_WHL && \
       pip install $TASKD_WHL
    
    
   # 环境变量
   ENV HCCL_WHITELIST_DISABLE=1
    
   # 创建/lib64/ld-linux-aarch64.so.1
   RUN umask 0022 && \
       if [ ! -d "/lib64" ]; \
       then \
           mkdir /lib64 && ln -sf /lib/ld-linux-aarch64.so.1 /lib64/ld-linux-aarch64.so.1; \
       fi
    
   # 增加安装任务调度依赖库
   RUN pip install apscheduler
    
    
   RUN rm -rf tmp && \
       rm -f $TOOLKIT && \
       rm -f $KERNEL && \
       rm -f $MINDX_ELASTIC_WHL && \
       rm -f $MINDIO_TTP_WHL && \
       rm -f $MINDSPORE_REQUIREMENTS && \
       rm -f $MINDSPORE_WHL
   ## 最后打包成镜像mindformers-dl:v1

3. 构建镜像。执行以下命令生成镜像。为了使Dockerfile更加安全，用户可以根据业务在其中定义HEALTHCHECK检查。通过在容器内部运行HEALTHCHECK [OPTIONS] CMD命令来检查容器的运行状况。注意不要遗漏命令结尾的“.”。

   docker build -t mindformers-dl:v1 .