Tool Deployment Architecture

This section describes the deployment architecture of AMCT in different product forms.

Product Models

The deployment architecture of AMCT varies according to the product form. The following describes the product form and then describes the deployment architecture of AMCT in different product forms.

The PCIe working mode of the Ascend AI Processor is used for distinguishing:

If the PCIe works in active mode and supports peripherals, it is called the Ascend RC scenario.
The CPUs of such products run the AI service software specified by the running user directly and connect to peripherals such as network cameras, I²C sensors, and SPI monitors as slave devices.
If the PCIe works in standby mode, it is called the Ascend EP scenario.
In the Ascend EP scenario, the host acts as the master, the device acts as the slave, and the customer's AI applications run on the host. The product, as a device, connects to the host over the PCIe interface, while the host loads AI tasks to the Ascend AI Processor on the device over the PCIe interface. The concepts of host and device are described as follows:
- Host: an x86 server or an Arm server connected to the hardware powered by an Ascend AI Processor. It leverages the neural network (NN) compute capability provided by the Ascend AI Processor.
- Device: a hardware backend powered by an Ascend AI Processor. It provides the server with the NN compute capability over the PCIe interface.

The working modes of the Ascend AI Processor and the supported products are as follows:

The Atlas 200I/500 A2 inference product has two working scenarios: Ascend EP and Ascend RC .
- Products supporting the Ascend RC scenario: Atlas 200 AI accelerator module and Atlas 200 DK
- Products supporting the Ascend EP scenario: Atlas 500 A2 edge station, Atlas 200I DK A2, and Atlas 200I A2 accelerator module
The following products support only the Ascend EP scenario:
- Atlas inference product : Atlas 300I Pro inference card
- Atlas training product : Atlas 800 training server and Atlas 300T training card
- Atlas A2 training product / Atlas A2 inference product : Atlas 800T A2 training server, Atlas 900 A2 PoD cluster basic unit, and Atlas 200T A2 Box16 heterogeneous subrack
- Atlas A3 training product / Atlas A3 inference product : Atlas 800T A3 supernode server, Atlas 900 A3 SuperPoD, and Atlas 800I A3 supernode server
- Atlas 350 Accelerator Card

Figure 1 shows the products and architectures of the two scenarios.

Figure 1 Ascend RC and Ascend EP scenarios

Environment Setup in the Ascend EP Scenario

Figure 2 shows the environment setup for AMCT. For details about the supported OSs, see Supported OSs. Before running inference, use ATC to convert the quantized model into an offline model adapted to the AI processor.

Figure 2 Environment setup in Ascend EP mode

Deploy AMCT on the server that meets the requirements to compress the model.
Use ATC to convert the compressed model into an offline model adapted to the AI processor.
Run inference on the resultant .om offline model obtained in 2 on the server powered by the AI processor.

For PyTorch models that require NPU-based quantization, AMCT installation on an environment with the Ascend AI processor is mandatory (both the driver/firmware and CANN package must be installed). For details, see Checking the OS Requirements and Environment.

Environment Setup in the Ascend RC Scenario

Figure 3 shows the AMCT deployment architecture. For details about the version mapping, see Checking the OS Requirements and Environment. Before running inference, use ATC to convert the quantized model into an offline model adapted to the AI processor.

Figure 3 Environment setup in Ascend RC mode

Install AMCT on an Ubuntu (AArch64) server to perform model compression.
Use ATC to convert the compressed model into an offline model adapted to the AI processor.
Run inference on the resultant .om offline model obtained in 2 on the server powered by the AI processor.

Parent topic: Overview