Overview
Technical Background and Benefits
As large language models evolve to be deployed in clusters with tens of thousands of devices, traditional collective communication libraries face the following challenges:
- The built-in communication algorithms cannot achieve optimal performance in all scenarios.
- Fine-grained pipeline parallelism is required in communication-compute fusion scenarios to effectively reduce the latency of compute and communication and improve the overall efficiency.
The closed black-box design of traditional collective communication libraries constrains researchers from exploring new communication primitives. Therefore, HCCL opens underlying communication capabilities and provides lightweight communication operator development APIs, achieving full-stack programmability of communication operators and facilitating communication solution innovation.
HCCL communication operator development APIs have the following key features:
- Supports multiple communication engines on Ascend devices to fully utilize hardware capabilities.
- Supports multiple communication protocols, including PCIe, HCCS, UB, and RoCE.
- The communication platform capability is decoupled from the communication operator development, and communication operators can be developed independently.
Supported Products
The custom development of communication operators supports the following products: