Interprocess communication

Any device memory, event resource, or Notify resource created by a host thread can be directly referenced by other threads in the same process. However, these pointers or handles are invalid outside the process and therefore cannot be directly used by threads of other processes.

To share device memory, event resources, or Notify resources between different processes, the application needs to use the inter-process communication APIs provided by the Runtime module to implement the following typical scenario: A primary process generates a batch of input data and provides the data to multiple secondary processes without regenerating or copying the data in each process. The required inter-process communication (IPC) API varies with the resource type. For details, see subsequent call examples.

Note that when the aclrtMalloc API is used to allocate device memory, the memory may be allocated from a larger underlying memory block to ensure performance. In this case, the IPC API checks whether the shared memory is page-table aligned. If not, the API intercepts the operation and reports an error to prevent information leakage caused by cross-process memory mapping. Therefore, you are advised to use the aclrtMalloc API to allocate memory based on memory allocation rules. The page table size varies with the type of memory being allocated. The page table size for normal page memory is 4 KB, while the page table size for huge page memory can be 2 MB or 1 GB.

Inter-Process Memory Sharing

The following uses process A (memory lender) and process B (memory borrower) as an example to describe the process of calling the API for sharing memory between two processes.

The following is the sample code for sharing memory between processes A and B, which is for reference only and cannot be directly copied for compilation and running. For details about the complete sample code, click here.

Allocate memory in process A and generate a shared key.

        
             uint keyLen = 65;
char[keyLen] key; 
void *ptrA = nullptr;

aclrtSetDevice(0);  // Process A uses device 0.
aclrtMalloc(&ptrA, size); 

aclrtIpcMemGetExportKey(ptrA, size, key, keyLen, ACL_RT_IPC_MEM_EXPORT_FLAG_DISABLE_PID_VALIDATION);  

// Exchange keys for cross-process communication (by writing files).
writeFile("file/ipc_mem", key, keyLen);

// Read and write the shared memory.
......

// After the shared memory is used, the lender disables the IPC memory sharing.
aclrtIpcMemClose(key);   
aclrtFree(ptrA);
aclrtResetDeviceForce(0);

In process B, import the shared memory using the shared key.

        
             uint keyLen = 65;
char[keyLen] key;
void *ptrB;

// Exchange keys for cross-process communication (by writing files).
readFile("file/ipc_mem", key, keyLen);

aclrtSetDevice(1);  // Process B uses device 1.

// Processes A and B use different devices. To import the shared memory for access, data exchange between the two devices must be enabled.
aclrtIpcMemImportByKey(&ptrB, key, ACL_RT_IPC_MEM_IMPORT_FLAG_ENABLE_PEER_ACCESS);  

// Read and write the ptrB memory.
......

// After use, the borrower disables the IPC memory sharing.
aclrtIpcMemClose(key);    
aclrtResetDeviceForce(1);

Inter-Process Event Sharing

Events can be synchronized between processes through inter-process event sharing. The following is the sample code for task synchronization between two processes. In this sample code, process A creates an event and shares it with process B. The sample code is for reference only and cannot be directly copied for compilation and running.

Create an event in process A and generate a shared handle.

        
             aclrtEvent event;
aclrtStream stream;
aclrtIpcEventHandle handle;

aclrtSetDevice(0);  // Process A uses device 0.
aclrtCreateEventExWithFlag (&event, ACL_EVENT_IPC);  // Create an IPC event.
aclrtCreateStream(&stream);  // Create a stream.

// Export the IPC sharing handle.
aclrtIpcGetEventHandle(event, &handle);

// Exchange keys for cross-process communication (by writing files).
writeFile("file/ipc_event", handle, ACL_IPC_EVENT_HANDLE_SIZE);

// Deliver a record task.
aclrtEventRecord(event, stream);  

// Destroy the shared event after it is used.
aclrtDestroyEvent(event);
aclrtResetDeviceForce(0);

In process B, import the shared event using the shared handle.

        
             aclrtStream stream;
aclrtEvent event;
aclrtIpcEventHandle handle;

aclrtCreateStream(&stream);  // Create a stream.
// Obtain the shared handle across processes (by writing files).
readFile("file/ipc_event", handle, ACL_IPC_EVENT_HANDLE_SIZE);

aclrtSetDevice(1);  // Process B uses device 1.

// Import the handle and return the shared event.
// Processes A and B use different devices.
aclrtIpcOpenEventHandle(handle, &event); 

// Deliver a wait task.
aclrtStreamWaitEvent(stream, event); 

// Synchronize tasks in the stream.
aclrtSynchonizeStream(stream);

// Destroy the shared event after it is used.
aclrtDestroyEvent(event);
aclrtResetDeviceForce(1);

Inter-Process Notify Sharing

Notifications can be implemented between processes through inter-process Notify sharing. The following is the sample code for task synchronization between two processes. In this sample code, process A creates a Notify and shares it with process B. The sample code is for reference only and cannot be directly copied for compilation and running. For the complete sample code, click here.

Note: The creator allocates Notify hardware resources. Therefore, only the creator's hardware can execute the wait task. For this, sharing a Notify has the following restriction: The aclrtNotifyWait API can be called only on the creator side to perform the wait task.

Create a Notify in process A and generate a shared key.

        
             uint keyLen = 65;
char key[keyLen]; 
aclrtNotify notify;
aclrtStream stream;

// Process A uses device 0.
aclrtSetDevice(0);  
aclrtCreateStream(&stream); 

aclrtNotifyCreate(&notify); 
// Export the key (that is, the shared Notify name).
aclrtNotifyGetExportKey(notify, key, keyLen, ACL_RT_NOTIFY_EXPORT_FLAG_DISABLE_PID_VALIDATION); 
// Exchange keys for cross-process communication (by writing files).
writeFile("file/ipc_notify", key, keyLen);

// Deliver a wait task.
aclrtNotifyWait(notify, stream);  

// Destroy the shared Notify after it is used.
aclrtNotifyDestroy(notify);
aclrtResetDeviceForce(0);

In process B, import the shared Notify using the shared key.

        
             uint keyLen = 65;
char key[keyLen];
aclrtNotify notify;
aclrtStream stream;

// Exchange keys for cross-process communication (by writing files).
readFile("file/ipc_notify", key, keyLen);

// Process B uses device 1.
aclrtSetDevice(1);  
aclrtCreateStream(&stream);

// Import the key and return the shared Notify.
// Processes A and B use different devices. To import the shared Notify, data exchange between the two devices must be enabled. You are advised to use ACL_RT_NOTIFY_IMPORT_FLAG_ENABLE_PEER_ACCESS.
aclrtNotifyImportByKey(&notify, key, ACL_RT_NOTIFY_IMPORT_FLAG_ENABLE_PEER_ACCESS); 

// Deliver a record task.
aclrtNotifyRecord (notify, stream); 

// Destroy the shared Notify after it is used.
aclrtNotifyDestroy(notify);
aclrtResetDeviceForce(1);

Inter-Process Memory Sharing Using VMM APIs

In addition to memory sharing using IPC Mem, Runtime also provides another set of memory management and sharing APIs. The Virtual Memory Management (VMM) APIs provide more flexible functions, including virtual address allocation, physical memory allocation, cross-process physical memory sharing, and mapping between virtual addresses and physical memory.

The following uses processes A and B as an example to describe the sample code for physical memory sharing between two processes on a device. The sample code is for reference only and cannot be directly copied for compilation and running. For details about the complete sample code, click here.

If processes A and B use different devices, you also need to call the aclrtDeviceEnablePeerAccess API to enable cross-device data exchange. For details, see Cross-Device Data Exchange.

In process A:

         
          
            
            
              // Query the memory allocation granularity.
const size_t dataSize = 1024 * sizeof(float);
aclrtPhysicalMemProp prop = {};
prop.handleType = ACL_MEM_HANDLE_TYPE_NONE;
prop.allocationType = ACL_MEM_ALLOCATION_TYPE_PINNED;
prop.location.type = ACL_MEM_LOCATION_TYPE_DEVICE;
prop.location.id = 0;
prop.memAttr = ACL_HBM_MEM_NORMAL;
size_t granularity = 0UL;
aclrtMemGetAllocationGranularity(&prop, ACL_RT_MEM_ALLOC_GRANULARITY_MINIMUM, &granularity);

// Allocate physical memory based on the memory allocation granularity.
size_t alignedSize = ((dataSize + granularity - 1U) / granularity) * granularity;
aclrtDrvMemHandle handle = nullptr;
aclrtMallocPhysical(&handle, alignedSize, &prop, 0);

// Reserve virtual memory.
void *virPtr;
aclrtReserveMemAddress(&virPtr, alignedSize, 0, nullptr, 0);

// Map the virtual memory to the physical memory.
aclrtMapMem(virPtr, alignedSize, 0, handle, 0);
aclrtMemAccessDesc desc = {};
desc.flags = ACL_RT_MEM_ACCESS_FLAGS_READWRITE;
desc.location.id = 0;
desc.location.type = ACL_MEM_LOCATION_TYPE_DEVICE;
aclrtMemSetAccess(virPtr, alignedSize, &desc, 1);

// Use virPtr to perform operations such as copy, read, and write.
......

// Import the shared handle.
uint64_t shareableHandle = 0ULL;
aclrtMemExportToShareableHandle(handle, ACL_MEM_HANDLE_TYPE_NONE, ACL_RT_VMM_EXPORT_FLAG_DISABLE_PID_VALIDATION , &shareableHandle);

// Pass the shared handle to process B.
writeFile("file/vmm_mem", shareableHandle, sizeof(shareableHandle));

// Unmap the virtual memory from the physical memory.
aclrtUnmapMem(virPtr);
// Free the virtual memory and physical memory.
aclrtReleaseMemAddress(virPtr);
aclrtFreePhysical(handle);

             

           

         
        

In process B:

         
          
            
            
              uint64_t shareableHandle = 0ULL;
// Obtain the shared handle from the file.
readFile("file/vmm_mem", &shareableHandle, sizeof(shareableHandle));

aclrtDrvMemHandle handle = nullptr; 
int32_t deviceId=0;  
aclrtMemImportFromShareableHandle(shareableHandle, deviceId, &handle);
// Query the memory allocation granularity.
const size_t data_size = 1024 * sizeof(float);
aclrtPhysicalMemProp prop = {};
prop.handleType = ACL_MEM_HANDLE_TYPE_NONE;
prop.allocationType = ACL_MEM_ALLOCATION_TYPE_PINNED;
prop.location.type = ACL_MEM_LOCATION_TYPE_DEVICE;
prop.location.id = 0;
prop.memAttr = ACL_HBM_MEM_NORMAL;
size_t granularity = 0UL;
aclrtMemGetAllocationGranularity(&prop, ACL_RT_MEM_ALLOC_GRANULARITY_MINIMUM, &granularity);
size_t alignedSize = ((dataSize + granularity - 1U) / granularity) * granularity;

// Reserve virtual memory based on the memory allocation granularity.
void *virPtr = nullptr;
aclrtReserveMemAddress(&virPtr, alignedSize, 0, nullptr, 0);
// Map the virtual memory to the physical memory.
aclrtMapMem(virPtr, alignedSize, 0, handle, 0);
aclrtMemAccessDesc desc = {};
desc.flags = ACL_RT_MEM_ACCESS_FLAGS_READWRITE;
desc.location.id = 0;
desc.location.type = ACL_MEM_LOCATION_TYPE_DEVICE;
aclrtMemSetAccess(virPtr,alignedSize, &desc, 1);

// Use virPtr to perform operations such as copy, read, and write.
......

// Unmap the virtual memory from the physical memory.
aclrtUnmapMem(virPtr);       
// Free the virtual memory and physical memory.
aclrtReleaseMemAddress(virPtr);
aclrtFreePhysical(handle);

             

           

         
        

Parent topic: Application Development (C&C++)