Overall Process

The process of developing an HCCL communication operator is as follows:

1. Define operator APIs: Define communication operator APIs based on the functions of the communication operator.
2. Query topology information: Obtain the communicator information such as the network layer and number of nodes.
3. (Optional) Select algorithms: To maximize communication performance, different algorithms are often deployed depending on the topology and data volume. Therefore, the optimal communication algorithm must be selected based on topology information.
4. Create resources: Allocate resources such as memory, threads, Notify, and channels required by the communication algorithm.
5. Orchestrate tasks: Use communication resources to coordinate ranks for synchronization and data movement to complete collective communication operations.
6. Dispatch the operator: Dispatch the operator kernel to the communication engine for execution.