SetScheduleMode

Function Usage

Sets the scheduling mode of operators when they are executed on NPUs.

Prototype

ge::graphStatus SetScheduleMode(const uint32_t schedule_mode)

Parameters

Parameter	Input/Output	Description
schedule_mode	Input	0: normal mode. The system schedules operator compute tasks to idle AI Cores. For example, if an AI Core operator requires four cores and an idle AI Core is available, the operator starts execution and one of its compute slices is scheduled to the idle core. Other compute slices are computed only when other idle cores become available. The normal mode is used by default. 1: batch mode. An operator is scheduled for computation only when there are sufficient idle AI Cores. For example, if an AI Core operator requires four AI Cores, the operator is executed only when four AI Cores are available.

Parameter

Input/Output

Description

schedule_mode

Input

0: normal mode. The system schedules operator compute tasks to idle AI Cores. For example, if an AI Core operator requires four cores and an idle AI Core is available, the operator starts execution and one of its compute slices is scheduled to the idle core. Other compute slices are computed only when other idle cores become available. The normal mode is used by default.

1: batch mode. An operator is scheduled for computation only when there are sufficient idle AI Cores. For example, if an AI Core operator requires four AI Cores, the operator is executed only when four AI Cores are available.

Returns

ge::GRAPH_SUCCESS on success.

ge::GRAPH_FAILED on failure.

For details about the definition of graphStatus, see ge::graphStatus.

Constraints

When the multi-core synchronization function is used, if the operator runs in multi-stream scenarios, the batch mode must be enabled by calling SetScheduleMode(1). Otherwise, a deadlock occurs, causing the process to be suspended.

In multi-stream scenarios, if the SyncAll operator of a stream is allocated to n physical cores and other streams preempt m cores (m < n), the batch mode is not enabled, the following problem occurs: The first n–m cores wait for the m cores to complete multi-core synchronization, but the m cores have not been started because their resources are preempted. As a result, a deadlock occurs.

Examples

ge::graphStatus TilingForAdd(TilingContext *context) {
  uint32_t batch_mode = 1U;
  auto ret = context->SetScheduleMode(batch_mode);  
  GE_ASSERT_SUCCESS(ret);
  ...
}

Parent topic: TilingContext