Deterministic Scheduling Framework and OpenHiva
Challenges
During robot application development, if the service function of the robot is regarded as a function f, the input of the sensor is used as the input parameter x, and the output of the robot is the result y. The requirement of the deterministic scheduling system is that "for all x, the execution time of f needs to be less than a fixed upper limit time t, the output result y is correct, and the smaller the t, the better". However, in an actual case, the service function f is complex, and many heterogeneous computing resources need to be managed, including a CPU, AIC (AI Core), AIV (AI Vector Core), a DVPP, and the like. In addition, multiple neural network models and service processes need to be supported, and a large amount of service code needs to be processed. Therefore, only a scheduling policy from a macro perspective can be used to ensure determinism.
- Deterministic compute resources: Reserve sufficient CPU compute resources and AIC, AIV, and DVPP hardware resources to ensure that service functions that meet the specifications can be implemented.
- Deterministic scheduler: In any scenario, the output time (usually at the microsecond level) of the result generated by one event scheduling is less than the maximum value.
- Deterministic service execution: With the same input, the execution time of any function is fixed, usually at the millisecond level.
Architecture and Principles
The following figure shows the principle of the deterministic scheduling system. The deterministic scheduling system consists of the shared memory, deterministic scheduling framework, and OpenHiva scheduling framework. The OpenHiva scheduling framework is the upper-layer programming framework, and the deterministic scheduling framework is the lower-layer programming framework. OpenHiva encapsulates complex underlying deterministic scheduling framework interfaces and AscendCL interfaces and provides a set of ROS-like communication mechanisms to present the interfaces as publish/subscription interfaces for developers to use.
The following figure shows the implementation process of the deterministic scheduling system.
- Shared memory:
- Data can be shared across processes on the entire data plane.
- Groups can be distinguished based on the sharing scope. Data can be shared in the same group, but cannot be accessed across groups.
- A process can belong to different groups so that data can be transmitted between different groups. (Currently, all shared memories use the same group, and users are unaware of shared memory groups. In future, different groups will be added for different processes to ensure function security.)
- Deterministic event scheduling: Event scheduling is the core of deterministic scheduling framework.
- All events of the driver data plane can be processed, including hardware events such as Hardware Task Scheduler (HWTS) and DVPP, and software events related to queues and timers.
- Two-level priority scheduling policies based on the process and event priority and non-preemption scheduling are supported. To ensure the determinism of task processing, the round robin and preemption scheduling modes are not supported.
- Deterministic queue scheduling: The main function of queue scheduling is to transfer the data pointer from the publisher queue to the subscriber queue based on the publish/subscribe relationship. In this way, the publisher and subscriber can be decoupled to define their own queue policies.
- DataMaster: It maintains the relationship between publishers and subscribers, maps the publish/subscribe relationship to the relationship between queues, and delivers the relationship for queue scheduling. The DataMaster needs to work with the OpenHiva scheduling framework.