Image Creation
For the MindSpore-based Pangu model, you need to create an image that adapts to the Pangu model by referring to this section.
Prerequisites
Obtain required software packages of the corresponding OS and Dockerfiles and script files required for packaging images. For details, see Table 1. {version} in the name of each software package indicates the version number.
Package |
Required (Yes/No) |
Description |
How to Obtain |
|---|---|---|---|
mindformers-{version}-py3-none-any.whl |
Yes |
MindSpore Transformers suite, a development suite that covers the entire process of training, fine-tuning, evaluation, inference, and deployment of a foundation model. The master version of MindSpore is the r0.3 branch. |
|
Dockerfile |
Yes |
Required for creating an image. |
Prepared by yourself. |
To avoid using a software package that has been tampered with during transmission or storage, download its digital signature file for integrity check while downloading the software package.
After the software package is downloaded from the Support website, verify its PGP digital signature by referring to the OpenPGP Signature Verification Guide. If the verification fails, do not use the software package, and contact Huawei technical support.
Before using software for installation or upgrade, verify the digital signature to ensure that the software has not been tampered with.
Carrier users: Visit https://support.huawei.com/carrier/digitalSignatureAction.
Enterprises: Visit https://support.huawei.com/enterprise/en/tool/pgp-verify-TL1000000054.
This section uses Ubuntu as an example.
Procedure
- Log in to the server as the root user.
- Upload the MindFormers source code to any directory (for example, /home/test) on the server.
- Perform the following steps to create a Dockerfile.
- Go to the directory where the software package is stored and run the following command to create a Dockerfile:
vi Dockerfile
- Write the content to the Dockerfile and run the :wq command to save the content. For details, see the Dockerfile compilation sample.
- Go to the directory where the software package is stored and run the following command to create a Dockerfile:
- Go to the directory where the software package is stored and run the following command to create a container image. Do not omit the period (.) at the end of the command.
docker build -t [OPTIONS] Image name_System architecture:Image tag .
For example:docker build -t test_train_arm64:v1.0 .
Table 2 describes parameters in the command.
Table 2 Parameters in the command Parameter
Description
-t
Image name.
OPTIONS
--disable-content-trust: ignores verification. This option is enabled by default. For security purposes, you are advised to disable this option.
Image name_System architecture:Image tag
Image name and tag. Change them based on the actual situation.
If Successfully built xxx is displayed, the image has been created.
- After the image is created, run the following command to view the image information:
docker images
Command output:
REPOSITORY TAG IMAGE ID CREATED SIZE test_train_arm64 v1.0 d82746acd7f0 27 minutes ago 749MB
Compilation Examples
Modify the software package version and architecture based on the actual situation.
- Dockerfile compilation sample
- Dockerfile example for the Ubuntu ARM OS
FROM xxx # Base training image ARG MINDFORMERS_PKG=mindformers-{version}-py3-none-any.whl WORKDIR /tmp COPY . ./ ENV http_proxy xxx ENV https_proxy xxx # Configure the Python pip mirror. RUN mkdir -p ~/.pip \ && echo '[global] \n\ index-url=https://pypi.doubanio.com/simple/\n\ trusted-host=pypi.doubanio.com' >> ~/.pip/pip.conf # Install MindFormers. RUN pip install $MINDFORMERS_PKG ENV http_proxy "" ENV https_proxy "" - Dockerfile example for the Ubuntu x86_64 OS
FROM xxx # Base training image ARG MINDFORMERS_PKG=mindformers-{version}-py3-none-any.whl WORKDIR /tmp COPY . ./ ENV http_proxy xxx ENV https_proxy xxx # Configure the Python pip mirror. RUN mkdir -p ~/.pip \ && echo '[global] \n\ index-url=https://pypi.doubanio.com/simple/\n\ trusted-host=pypi.doubanio.com' >> ~/.pip/pip.conf # Install MindFormers. RUN pip install $MINDFORMERS_PKG ENV http_proxy "" ENV https_proxy ""
To make Dockerfiles more secure, you can define HEALTHCHECK in them based on services. Run the HEALTHCHECK [OPTIONS] CMD command in the container to check the running status of the container.
- Dockerfile example for the Ubuntu ARM OS