Important Notes
- A stream is a concept on the device. Multi-stream parallelism is the parallel running on the device.
- If Wait is performed before Record is performed, you are advised to use aclrtCreateEventWithFlag() to create an aclrtEvent whose flag is ACL_EVENT_SYNC.
- A deadlock occurs if EventOperation (WaitOp) with OperatorType set to WAIT, the stream synchronization API (aclrtSynchronizeStream), and EventOperation (RecordOp) with OperatorType set to RECORD are called in sequence on a thread. This is because when WaitOp is called first, EVENT_WAIT is delivered to the device to block the current stream until RecordOp is called. However, the stream synchronization API is called between the two operations. As a result, the host is blocked until the task on the corresponding stream is complete, causing the host and device to be interlocked.
- Do not use aclrtSynchronizeDevice when inter-stream synchronization is used. Otherwise, the device and host are interlocked because this API blocks the host until all tasks on all streams on the device are complete.
- Ensure that communication operators on different streams are not executed concurrently.
- When Cube operators or Vector operators are executed concurrently on different streams, the operator execution duration is prolonged or the operators run abnormally because computing resources are preempted.
- The ATB environment variable ATB_STREAM_SYNC_EVERY_OPERATION_ENABLE cannot be set to 1 for single-thread calling.
Parent topic: Inter-graph Synchronization