MindIE
- Large language model (LLM) inference scenario:
- Using the open-source inference engine vLLM: Ascend supports open-source inference engines, such as vLLM, to accelerate inference.
- MindIE text generation: MindIE LLM enables text generation inference and accelerates the E2E serving inference process.
- Multimodal inference scenario:
- MindIE vision generation: MindIE SD enables high-performance vision generation inference.
- Release Notes
Describes the version mapping, new features, modified features, deleted features, known issues, resolved issues, and fixed vulnerabilities in the current MindIE version.
- Quick Start
Describes the overall architecture of MindIE and how to get started with model inference journeys.
- List of the Supported Models
Describes the models supported by MindIE.
Using the vLLM Open-Source Inference Engine
- Getting started with vLLM text generation inference
Describes how to use the MindIE Turbo acceleration plugin on an Atlas 800I A2 inference server to quickly use vLLM for text generation inference.
- Model list supported by vLLM Ascend
Contains models supported by vLLM Ascend.
- vLLM and vLLM Ascend installation guide
Provides guidance for installing vLLM and vLLM Ascend to help users manually set up the development environment.
- MindIE Turbo Development Guide
Describes the installation, usage, and features of MindIE Turbo, an acceleration plugin library for LLM inference engines, which is developed by Huawei based on Ascend hardware.
MindIE Text Generation
- Getting started with MindIE text generation inference
Describes how to use MindIE LLM on the Atlas 800I A2 inference server to quickly perform text generation inference.
- List of the Supported Models
Contains LLMs supported by MindIE.
- MindIE Installation Guide
Describes how to install, upgrade, and uninstall MindIE.
- MindIE LLM Development Guide
Describes the model inference process, features such as quantization, long sequence, and multi-machine functions, as well as application programming interface (API) usage. MindIE LLM is a large language model inference component of MindIE.
- MindIE Motor Development Guide
Describes the inference serving capabilities of MindIE Motor, functions of each component, and application scenarios. MindIE Motor is an inference serving framework designed for general-purpose model scenarios, including MindIE Service Tools, MindIE Client, and cluster management components (Deployer, Controller, and Coordinator).
MindIE Vision Generation
- Getting started with MindIE vision generation inference
Describes how to use MindIE SD to conduct a text-to-video process on the Atlas 800I A2 inference server.
- List of the Supported Models
Contains multimodal generation models supported by MindIE.
- MindIE Installation Guide
Describes how to install, upgrade, and uninstall MindIE.
- MindIE SD Development Guide
Describes the vision generation function and API reference of MindIE SD, which is a vision generation inference model suite of MindIE.