Orchestrating Tasks

Procedure

Task orchestration is a process in which ranks involved in collective communication coordinate and perform synchronization and data movement in an orderly manner to complete a collective communication operation. The main purpose of task orchestration is to execute tasks on different communication threads in parallel to maximize resource utilization and improve overall performance.

Task orchestration includes the following steps:

1. Obtain the local communication memory, which is called HCCL buffer in HCCL.
2. Copy the operator input data to the HCCL buffer. The common data plane API is HcommLocalCopyOnThread.
   

   The HCCL buffer is a block of pinned memory on a device managed by each HCCL communicator. Communication tasks are executed asynchronously. To ensure that the user input data is still valid when the communication task is actually executed, the input data needs to be copied to the HCCL buffer with a fixed memory address.

3. Split the input data and calculate the offset.

   The default size of the HCCL buffer is 200 MB. If the size of the input data exceeds this size, the input data needs to be split into multiple data blocks for processing.

4. Perform pre-sync. The primary thread notifies the secondary thread to start the task, and the secondary thread waits for the notification from the primary thread. Common data plane APIs are HcommThreadNotifyWaitOnThread and HcommThreadNotifyRecordOnThread.
5. Copy the remote data to the local HCCL buffer. Common data plane APIs are HcommReadOnThread and HcommReadReduceOnThread.
6. Post-sync: The secondary thread notifies the primary thread that the task is complete, and the primary thread waits for the notification from the secondary thread.
7. Copy the result data in the HCCL buffer to the operator output memory. The common data plane API is HcommLocalCopyOnThread.

Sample Code

• The following uses the custom Send operator as an example to describe its task orchestration code snippet on the AICPU:

  uint64_t size = count * dataTypeSize;  // Data size = Number of data records × Data type size
  // 1. Copy the data to the transit memory.
  HcommLocalCopyOnThread(threadHandle, localAddr, inputPtr, size);
  // 2. Notify the RX end that the local end has prepared the data.
  HcommChannelNotifyRecordOnThread(threadHandle, channelHandle, 0);
  // 3. Wait for the RX end to notify that the local data has been read.
  HcommChannelNotifyWaitOnThread(threadHandle, channelHandle, 1, 1800);

• The following uses the custom Receive operator as an example to describe its task orchestration code snippet on the AICPU:

  // 1. Wait for the TX end to notify that the local end can start to read data.
  HcommChannelNotifyWaitOnThread(threadHandle, channelHandle, 0, 1800);
  // 2. Read data from the TX end.
  HcommReadOnThread(threadHandle, channelHandle, outputPtr, remoteAddr, size);
  // 3. Notify the TX end that the local end has read the data.
  HcommChannelNotifyRecordOnThread(threadHandle, channelHandle, 1);