User-built Container Deployment and Inference

This section describes how to create a container on a device in the edge inference scenario, upload an image, and decrypt the image.

1. Import the image package to the inference device. The image must contain Python related components.
2. Run the following command to import the image to the container. Ensure that the image meets the security requirements of your organization.

   docker load < cfs-infer.tar

   cfs-infer.tar indicates the name of the image package created in Creating a Container Image.

3. Save the seccomp_profile.json and cfs_profile files that meet the security standards of your organization to the current directory. You can obtain the files from the aiguard_plugin directory from the open source community.
   1. Parse the cfs_profile file.

      apparmor_parser -r -W cfs_profile

      The file parsing is not required in the Altas 500 device environment.

   2. Start the container. The following uses the Altas 500 Pro device as an example.

      docker run -it \
       --security-opt seccomp=seccomp_profile.json  \
       --security-opt apparmor=cfs_profile  \ 
       --restart=on-failure:5 \
       --device=/dev/fuse --net=aiguard  \
       --device=/dev/davinci0 \
       --device=/dev/davinci_manager \
       --device=/dev/devmm_svm \
       --device=/dev/hisi_hdc \
       -v /usr/local/Ascend/driver:/usr/local/Ascend/driver:ro \
       --cpus=10 -c 512 -m 8192m image name:tag    \
      unshare --propagation unchanged -m -r /bin/bash

      • --security-opt: configures security options.
      • --restart: restart policy of the container. Select on-failure and set the maximum number of restart times to 5.
      • -it (-i or -t): enables STDIN for console interaction and allocates TTY devices for terminal login. The default value is false.
      • --device: adds a host device to the container.
      • -v, --volume: mounts a storage volume to a directory of the container.
      • --cpus: number of used CPUs.
      • -c: CPU quota that can be used.
      • -m: maximum memory that can be used.
4. Write the pre-shared key created in Creating a Pre-shared Key and Binding It to the Master Key to the container and then execute inference.
   1. Write the ciphertext of the pre-shared key to /run/secrets/aiguard-preshared-key. Ensure that the owner of the /run/secrets directory is the running user.
   2. Execute inference. Before inference, you need to enter the password in the request example for creating a pre-shared key and the password used for encrypting the private key.

      /job/crypto_fs/bin/crypto_fs /job/encrypt /job/dec \
      --cert_file=/job/rsa.CFS.pem \
      --pri_key_file=/job/server.key \
      --ca_file=/job/rsa.trust.pem \     
      -p "/job/dec/infer/infer_start_cfs.py" 

      For details about the Crypto_fs parameters, see Crypto_fs Command Parameters.

      

      • If the crypto_fs process stops unexpectedly and will not be started later, uninstall the file system to which the decryption directory is mounted.

        umount /job/dec

      • If there are a large number of model files and the same working key is used for encryption, add the following parameters:

        /job/crypto_fs/bin/crypto_fs /job/encrypt /job/dec \
        --cert_file=/job/rsa.CFS.pem \
        --pri_key_file=/job/server.key \
        --ca_file=/job/rsa.trust.pem \     
        -p "/job/dec/infer/infer_start_cfs.py" \
        -t 60  # Key cache duration, which takes effect only during decryption.