Quick Start

The Multimodal SDK provides acceleration for multimodal preprocessing, including image decoding, resize/crop, video frame decoding, and audio loading. This document helps you set up the environment using Docker and run your first Python example.

For native installation on the host, see Installation Guide.

Prerequisites

Before you begin, make sure that the following requirements are met:

• Hardware: Atlas 800I A2 inference server. See Supported Hardware and Operating Systems.
• Docker: Docker is installed, and the current user can run containers.
• Test image: /data/test.jpg is provided in the image, so no additional directory needs to be mounted for the test image.

Step 1: Pulling the Image

1. Select the matching version

   • Visit the Ascend Community Image Repository.
   • Select the image version corresponding to your hardware model (Atlas 800I A2 inference server).
   • Only the aarch64 CPU architecture is currently supported.

2. Pre-check the environment

   • Run the following command to verify the NPU driver status:

   npu-smi info

   • Check the compatibility between the driver version and the CANN version in the image. For details, see Firmware and Driver.

3. Pull the image

   The image tag format is {version}-{cann}-{torch_npu}-910b-{os}-{python}-aarch64. The variables are described as follows:

   Variable	Description	Example Value
   {version}	Multimodal SDK version	26.1.0
   {cann}	CANN version	9.1.0
   {torch_npu}	torch_npu version	2.6.0.rc1
   {os}	Base OS	ubuntu22.04 / openeuler24.03
   {python}	Python version	py3.12

   TAG={version}-{cann}-{torch_npu}-910b-{os}-{python}-aarch64
   docker pull swr.cn-south-1.myhuaweicloud.com/ascendhub/multimodalsdk:${TAG}
   docker tag swr.cn-south-1.myhuaweicloud.com/ascendhub/multimodalsdk:${TAG} \
       multimodalsdk:${TAG}

   The following example uses version 26.1.0, Ubuntu 22.04, and Python 3.12:

   docker pull swr.cn-south-1.myhuaweicloud.com/ascendhub/multimodalsdk:26.1.0-cann9.1.0-torch_npu2.6.0.post5-910b-ubuntu22.04-py3.12-aarch64
   docker tag swr.cn-south-1.myhuaweicloud.com/ascendhub/multimodalsdk:26.1.0-cann9.1.0-torch_npu2.6.0.post5-910b-ubuntu22.04-py3.12-aarch64 \
       multimodalsdk:26.1.0-cann9.1.0-torch_npu2.6.0.post5-910b-ubuntu22.04-py3.12-aarch64

Step 2: Starting the Container

First, check whether a container with the same name already exists. If it does, stop and remove it:

docker stop multimodal_container
docker rm multimodal_container

Run the following command to start the container and then check whether it started successfully:

docker run \
    --name multimodal_container \
    --device /dev/davinci0 \
    --device /dev/davinci_manager \
    --device /dev/devmm_svm \
    --device /dev/hisi_hdc \
    -v /usr/local/dcmi:/usr/local/dcmi \
    -v /usr/local/bin/npu-smi:/usr/local/bin/npu-smi \
    -v /usr/local/Ascend/driver/lib64:/usr/local/Ascend/driver/lib64 \
    -v /usr/local/Ascend/driver/version.info:/usr/local/Ascend/driver/version.info \
    -v /etc/ascend_install.info:/etc/ascend_install.info \
    -itd multimodalsdk:26.1.0-cann9.1.0-torch_npu2.6.0.post5-910b-ubuntu22.04-py3.12-aarch64 bash

docker ps -a | grep multimodal_container

Enter the container. Perform all subsequent operations inside the container.

docker exec -it multimodal_container bash

Step 3: Loading the Environment

export MULTIMODAL_SDK_HOME="/usr/local/multimodal"
source ${MULTIMODAL_SDK_HOME}/script/set_env.sh

Step 4: Running the Verification Script

If /data/test.jpg is not included in the image, download a test image to the container using the following command:

mkdir -p /data
wget --tries=3 --timeout=30 --waitretry=5 -O /data/test.jpg https://raw.gitcode.com/Ascend/MultimodalSDK/blobs/f1f648b7a8b8a67c7509b3425a89f743bbf59563/dog_1920_1080.jpg

export TEST_IMAGE="/data/test.jpg"
chmod 640 "$TEST_IMAGE"
python3 - <<'EOF'
import os
from mm import Image, DeviceMode, Interpolation

test_image = os.environ["TEST_IMAGE"]
img = Image.open(test_image, "cpu")
img_resize = img.resize((500, 500), Interpolation.BICUBIC, DeviceMode.CPU)
print(f"resize output shape: {img_resize.numpy().shape}")
EOF

If the following output is displayed, the verification is successful:

resize output shape: (500, 500, 3)

Step 5: Cleaning Up the Environment

After verification, exit the container. Clean up the container to release resources.

exit
docker stop multimodal_container
docker rm multimodal_container

What to Do Next

Quick Troubleshooting