Creating an Inference Image

Typical Service Scenarios

Some applications require the NPU computing power of edge devices. The following describes how to create a corresponding inference image, which is for reference only.

Prerequisites

• Prepare the container OS image by yourself.
• Prepare the offline inference engine package and service inference program package according to the following table.

Verifying the Digital Signature

To prevent a software package from being maliciously tampered with during transfer or storage, download also the corresponding digital signature file for integrity verification while downloading the software package.

After the software package is downloaded from the Huawei Support website, verify its PGP digital signature. For details, see the OpenPGP Signature Verification Guide. If the verification fails, do not use the software package, and contact Huawei technical support.

Before using or upgrading a software package, verify its digital signature to ensure that the software package is not tampered with.

For carrier users: http://support.huawei.com/carrier/digitalSignatureAction.

For enterprises, visit https://support.huawei.com/enterprise/en/tool/pgp-verify-TL1000000054.

Procedure

1. Upload the software packages and scripts to the same directory (for example, /home/test).
   • Ascend-cann-nnrt_{version}_linux-aarch64.run
   • Service inference program package
   • Dockerfile
   • slog.conf
   • install.sh
   • run.sh
2. Perform the following steps to create a Dockerfile.
   1. Log in as the root user and run the following commands in sequence to query and record the UIDs and GIDs (which will be used in the Dockerfile file) of the HwHiAiUser, HwBaseUser, and HwDmUser users in the image environment:

      id HwHiAiUser
      id HwBaseUser
      id HwDmUser

   2. Go to the software package upload directory in Step 1 and run the following command to create a Dockerfile:

      vi Dockerfile

   3. Enter the following content and run the :wq command to save the modification. (The Ubuntu Arm OS is used as an example, and the content is only an example. Perform secondary development as required.)

      # Container operating system and tag. Change them based on the actual situation.
      FROM ubuntu:22.04
      
      ARG NNRT_PKG
      ARG DIST_PKG
      # This document uses the /usr/local/Ascend directory as the NNRT installation directory. If you want to install the NNRT in another directory, change it to the target directory.
      ARG ASCEND_BASE=/usr/local/Ascend
      WORKDIR /home/AscendWork
      COPY $NNRT_PKG .
      COPY $DIST_PKG .
      COPY install.sh .
      
      # The inference program needs to use the bottom-layer driver. The running of the bottom-layer driver depends on three users, HwHiAiUser, HwDmUser, and HwBaseUser.
      # Create a user and user group for running the inference application. For example, the UIDs and GIDs of HwHiAiUse, HwDmUser, and HwBaseUser are 1000, 1101, and 1102, respectively.
      RUN umask 0022 && \
          groupadd  HwHiAiUser -g 1000 && \
          useradd -d /home/HwHiAiUser -u 1000 -g 1000 -m -s /bin/bash HwHiAiUser && \
          groupadd HwDmUser -g 1101 && \
          useradd -d /home/HwDmUser -u 1101 -g 1101 -m -s /bin/bash HwDmUser && \
          usermod -aG HwDmUser HwHiAiUser && \
          groupadd HwBaseUser -g 1102 && \
          useradd -d /home/HwBaseUser -u 1102 -g 1102 -m -s /bin/bash HwBaseUser && \
          usermod -aG HwBaseUser HwHiAiUser
         
      # Install NNRT and decompress the inference program.
      RUN chmod +x $NNRT_PKG && \
          echo y | ./$NNRT_PKG --quiet --install --install-path=$ASCEND_BASE \
          --install-for-all --force && \
          sh install.sh && \
          chown -R HwHiAiUser:HwHiAiUser /home/AscendWork/ && \
          rm $NNRT_PKG && \
          rm $DIST_PKG && \
          rm install.sh
      
      ENV LD_LIBRARY_PATH=/usr/local/Ascend/nnrt/latest/lib64:/usr/local/Ascend/driver/lib64:/usr/lib64
      ENV LD_PRELOAD=/lib/aarch64-linux-gnu/libc.so.6
      
      RUN ln -sf /lib /lib64 && \
          mkdir /var/dmp && \
          mkdir /usr/slog && \
          chown HwHiAiUser:HwHiAiUser /usr/slog && \
          chown HwHiAiUser:HwHiAiUser /var/dmp
      
      # Copy the log configuration file.
      COPY --chown=HwHiAiUser:HwHiAiUser slog.conf /etc
      
      COPY --chown=HwHiAiUser:HwHiAiUser run.sh /home/AscendWork/run.sh
      RUN chmod 640 /etc/slog.conf && \
          chmod +x /home/AscendWork/run.sh
      	
      # If you need to use the AI CPU operator, delete the comment tag (#) from the following commands:
      #RUN cp /usr/local/Ascend/nnrt/latest/opp/Ascend/aicpu/Ascend-aicpu_syskernels.tar.gz /home/HwHiAiUser/ && \
      #    rm -rf /usr/local/Ascend/nnrt/latest/opp/Ascend/aicpu/Ascend-aicpu_syskernels.tar.gz && \
      #    echo $(wc -c /home/HwHiAiUser/Ascend-aicpu_syskernels.tar.gz|awk ' {print$1} ') > /home/HwHiAiUser/aicpu_package_install.info && \
      #    tail -c +8449 /home/HwHiAiUser/Ascend-aicpu_syskernels.tar.gz > /home/HwHiAiUser/aicpu.tar.gz && \
      #    rm -rf /home/HwHiAiUser/Ascend-aicpu_syskernels.tar.gz && \
      #    chown HwHiAiUser:HwHiAiUser /home/HwHiAiUser/aicpu.tar.gz && \
      #    mkdir -p /home/HwHiAiUser/aicpu_kernels
      #RUN tar -xvf /home/HwHiAiUser/aicpu.tar.gz -C /home/HwHiAiUser/ 2>/dev/null;exit 0
      #RUN rm -rf /home/HwHiAiUser/aicpu.tar.gz && \
      #    mv /home/HwHiAiUser/aicpu_kernels_device/* /home/HwHiAiUser/aicpu_kernels/ && \
      #    chown -R HwHiAiUser:HwHiAiUser /home/HwHiAiUser/
      
      # If you do not need to use the AI CPU operator, delete the comment tag (#) from the following commands:
      #RUN rm -rf /usr/local/Ascend/nnrt/latest/opp/Ascend/aicpu/Ascend-aicpu_syskernels.tar.gz && \
      #    chown -R HwHiAiUser:HwHiAiUser /home/HwHiAiUser/
      
      USER 1000
      CMD bash /home/AscendWork/run.sh

   4. After creating the Dockerfile, run the following command to change the permission on the Dockerfile:

      chmod 600 Dockerfile

   5. Compilation example of install.sh:

      cd /home/AscendWork
      # Decompress the service inference program package based on the package format.
      tar -xzvf vpc_resnet50_imagenet_classification.tar.gz

      Compilation example of run.sh:

      #!/bin/bash
      mkdir /dev/shm/dmp
      nohup /var/dmp_daemon -I -M -U 8087 >&/dev/null &
      /var/slogd -d
      # Go to the directory where the executable file of the service inference programs is located. Use the actual path.
      cd /home/AscendWork/vpc_resnet50_imagenet_classification/out
      # Run the executable file. You need to modify the file according to the actual situation.
      ./main

3. Go to the directory where the software packages are stored and run the following command to create a container image:

   docker build -t image-name:tag --build-arg NNRT_PKG=nnrt-name --build-arg DIST_PKG=distpackage-name .

   If the message "Successfully built xxx" is displayed, the image is successfully built. For details about the command, see Table 2.

   

   Do not omit the period (.) at the end of the command.

   Table 2 Parameters in the commands

   Parameter	Description
   image-name:tag	Specifies the image name and tag. Set this parameter as required.
   NNRT_PKG	nnrt-name specifies the name of the offline inference engine package. Do not omit the file name extension. Replace it with the actual one.
   DIST_PKG	distpackage-name specifies the name of the service inference program package. Do not omit the file name extension. Replace it with the actual one.

4. Run the following command to save the container image to the drive:

   docker save image-name:tag |gzip -c > image-name.tar.gz