SyncAll

Applicability

Function Usage

When different AI Cores operate the same global memory block, this function can be called to synchronize the AI Cores to avoid data dependency problems such as write-after-read, read-after-write, and write-after-write. Currently, multi-core synchronization is classified into hardware synchronization and software synchronization. Hardware synchronization uses the full-core synchronization instruction of the hardware to ensure multi-core synchronization. Software synchronization is implemented through software algorithm simulation.

Prototype

• Soft synchronization:

  1 2	template <bool isAIVOnly = true> __aicore__ inline void SyncAll(const GlobalTensor<int32_t>& gmWorkspace, const LocalTensor<int32_t>& ubWorkspace, const int32_t usedCores = 0)

• Hard synchronization:

  1 2	template <bool isAIVOnly = true, const SyncAllConfig& config = DEFAULT_SYNC_ALL_CONFIG> __aicore__ inline void SyncAll()

Parameters

Returns

None

Constraints

• The space allocated for the gmWorkspace cache must be greater than or equal to the number of cores multiplied by 32 bytes, and the cache value must be initialized to 0. Currently, there are two common initialization modes.
  • Perform initialization on the host to ensure that the gmWorkspace cache has been initialized to 0 when this API is transferred.
  • Initialize the gmWorkspace cache during kernel initialization. Note that all gmWorkspace cache space needs to be initialized on each core.
• The space allocated for ubWorkspace must be greater than or equal to the number of cores multiplied by 32 bytes.
• For the Atlas 350 Accelerator Card, if the soft synchronization API is continuously called (polling) for multi-core synchronization, the bus will be occupied for a long time. As a result, other cores cannot access the global memory. There will be a deadlock when multiple cores read the same global memory. If polling is required, you are advised to add Nop instructions between the calls to this API. The recommended number of instructions is 800, which is generally the number of cores to be synchronized multiplied by 200.
• When this API is used for multi-core control, the logical numBlocks specified during operator calling must be less than or equal to the number of cores for running the operator. Otherwise, the framework inserts abnormal synchronization during multi-round scheduling, causing the kernel to stop responding.
• In separate mode, you are advised to use the hard synchronization API instead of the soft synchronization API. The soft synchronization API is applicable only to pure vector scenarios and has low performance. When using the hard synchronization API, you need to set the kernel type based on the scenario.
  • In pure vector or cube scenarios, set the kernel type to KERNEL_TYPE_MIX_AIV_1_0 or KERNEL_TYPE_MIX_AIC_1_0.
  • In vector and cube fused scenarios, set the kernel type based on the actual situation.

• When using this API, you are advised to enable the batch mode so that an operator exclusively occupies all required core resources. Otherwise, a deadlock may occur if the following conditions are met:
  • Multi-stream concurrency scenarios (≥ 2 execution streams)
  • Concurrent execution of at least two operators.
  • The total number of cores of all concurrent operators exceeds the number of physical cores.
  • Two or more concurrent operators use inter-core synchronization.

  Specifically, in a multi-stream scenario, although n physical cores are allocated to an inter-core synchronization operator of a stream, only (n – m) cores may be scheduled and executed first, and remaining m cores are not started because they are preempted by inter-core synchronization operators of other streams. When the first started (n – m) cores reach inter-core synchronization, they wait for the remaining m cores to complete execution. However, the remaining m cores are occupied by the inter-core synchronization operators of other streams and cannot be released, resulting in a deadlock.

  In kernel direct debugging scenarios, use the __schedmode__(mode) qualifier to set the batch mode. In project-based operator development scenarios, use the SetScheduleMode API of TilingContext to set the batch mode. For details, see Basic Data Structures and API List.

Example

In this example, eight cores are used for data processing. Each core processes 32 pieces of float-type data. The data is multiplied by 2 and then added to the data on other cores that are multiplied by 2 in the same way. The intermediate result is saved to workGm. Therefore, data needs to be synchronized between multiple cores. In this example, when software synchronization is used, the value of syncGm passed by the entrypoint function has been initialized to 0 on the host. If hardware synchronization is used in the following test cases, syncGm and workQueue do not need to be transferred.

...
// syncGlobal is the user-defined global space, which is used as the shared buffer for all cores. Its type is GlobalTensor. workLocal is the user-defined local space, which is used exclusively by each core. Its type is LocalTensor.
int srcDataSize = 256; // Number of elements involved in computation
int32_t blockNum = AscendC::GetBlockNum(); // Obtain the total number of cores.
int32_t blockIdx = AscendC::GetBlockIdx(); // Obtain the ID of the current working core.
uint32_t perBlockSize = srcDataSize / blockNum; // Each core evenly processes the same number of pieces of data.
// Save the data computed by the current working core to the external workspace. workGlobal is a GlobalTensor, and dstLocal is a LocalTensor.
AscendC::DataCopy(workGlobal[blockIdx * perBlockSize], dstLocal, perBlockSize);
// Wait until all cores complete the computation.
AscendC::SyncAll(syncGlobal, workLocal);
...