Concepts and Application Scenarios

Concept

Context is the core abstraction in CANN Runtime and represents the execution environment on a device. It encapsulates the runtime status of compute, memory, stream, and other resources on a device, and serves as the basic carrier for Runtime operations.

Each context is bound to a specific device and contains the following information on the device:

Compute resources: execution of kernel, system, and other tasks.
Memory resources: device memory allocation and management.
Stream resources: task queue and scheduling status.
Runtime configuration: parameters that affect task execution.

A context is bound to a thread, and a thread can use only one context at a time. When a Runtime API is executed, the context bound to the current thread is automatically used.

Why You Need Context

The core scenarios for using context are as follows:

Multi-threaded parallel computing: When multiple threads concurrently use the same device, each thread creates an independent context to prevent resource competition and status disorder.
Fine-grained resource control: It is necessary to control the timing of resource creation, use, and release in a more fine-grained manner.
Multi-threaded context sharing: The explicitly created context can be shared among multiple threads in a process.
Modular program design: Different functional modules use independent contexts to facilitate resource management and fault locating.

When the aclrtSetDevice API is called, Runtime automatically creates a default context for the specified device. For simple applications, default contexts are sufficient. However, for complex applications, explicitly creating and managing contexts have the following advantages.

**Table 1** Explicit Context vs. Default Context
Scenario	Default Context	Explicit Context
Multi-threaded programming	Threads share the default context, and the task execution sequence depends on thread scheduling.	Each thread has an independent context, facilitating isolation and debugging.
Resource isolation	Different modules on the same device share resources.	Different modules use independent contexts, resulting in clearer resource isolation.
Code maintainability	The context status is unclear when a thread switches to another device.	Explicit context specification makes the code's intent clear.
Resource lifecycle	The context is tied to the device lifecycle.	The context creation and destruction are controlled independently.

Relationship Among the Context, Device, and Stream

The following figure shows the relationship among the context, device, and stream.

Device: a physical NPU. Multiple contexts can be created on a device.
Context: the execution environment on a device, including the default stream and user-created streams.
Stream: a task queue, which belongs to a specific context.

Parent topic: Context Management