Cross-Device Data Exchange

Cross-device data exchange in this section refers to that devices can access the memory of each other within a process based on the hardware networking (for example, in a PCIe or HCCS interconnection topology). You can use the aclrtDeviceCanAccessPeer API to check whether data exchange is supported between two devices. If yes, call the aclrtDeviceEnablePeerAccess API based on the access direction to enable data exchange from one device to another. For example, call the aclrtDeviceEnablePeerAccess API once to enable data exchange from device 0 to device 1, and then call the aclrtDeviceEnablePeerAccess API again to enable data exchange from device 1 to device 0. To disable data exchange between devices, call aclrtDeviceDisablePeerAccess. For details about the communication between two processes, see Interprocess communication.

The following is the sample code for cross-device memory copy, which is for reference only and cannot be directly copied for compilation and running: For the complete sample code, click here.

aclInit (NULL); // Perform initialization.
int32_t canAccessPeer = 0;
aclrtDeviceCanAccessPeer(&canAccessPeer, 0, 1);      // Check whether data exchange is supported between device 0 and device 1.
if (canAccessPeer == 1) {
	aclrtSetDevice(0);                           // Perform operations on device 0.
	uint32_t reserveFlag = 0U;
	aclrtDeviceEnablePeerAccess(1, reserveFlag); // Enable data exchange from the current device (device 0) to the specified device (device 1).
	void *dev0Mem = nullptr;
	aclrtMalloc(&dev0Mem, 10, ACL_MEM_MALLOC_HUGE_FIRST_P2P);  
	aclrtSetDevice(1);                           // Perform operations on device 1.
	aclrtDeviceEnablePeerAccess(0, reserveFlag); // Enable data exchange from the current device (device 1) to the specified device (device 0).
	void *dev1Mem = nullptr;
	aclrtMalloc(&dev1Mem, 10, ACL_MEM_MALLOC_HUGE_FIRST_P2P);
	aclrtMemcpy(dev1Mem, 10, dev0Mem, 10, ACL_MEMCPY_DEVICE_TO_DEVICE); // Copy the memory data from device 0 to device 1.
	aclrtDeviceDisablePeerAccess(0);             // Disable data exchange from the current device (device 1) to the specified device (device 0).
	aclrtFree(dev1Mem);
	aclrtResetDevice(1);                         // Free resources of device 1.
        aclrtSetDevice(0);                           // Switch to device 0 to perform operations.
        aclrtDeviceDisablePeerAccess(1);             // Disable data exchange from the current device (device 0) to the specified device (device 1).
	aclrtFree(dev0Mem);
	aclrtResetDeviceForce(0);                    // Perform operations on device 0 and call aclrtResetDevice to free the resources of device 0.
	printf("P2P copy success\n");
} else {
	printf("current device doesn't support p2p feature\n");
}
aclFinalize(); // Perform deinitialization.

Parent topic: Multi-device programming