Documentation
Download

Development Process

This topic describes how to develop the RPing function.

Overall Process

The development process of the RPing function is as follows.

Figure 1 Development process of the RPing function

Procedure

  1. Initialize the RPing function and start network status monitoring.

    Call the HccnRpingInit API to initialize the RPing function for each NPU that participates in RPing monitoring. In a process, this API is called only once for each NPU.

    Before initializing the RPing function, you need to define the HccnRpingInitAttr object, initialize related information, and specify a device.

    • The sample code for Atlas 350 Accelerator Card is as follows:
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
          // Configure initialization information.
    int ipLen_ = 32;
    int devId = 1;
    HccnRpingInitAttr *initAttr = new HccnRpingInitAttr();
    initAttr->mode = HCCN_RPING_MODE_UB;  // Set the network type to UB.
    initAttr->port = 13886; // Listening port, which can be an unused port.
    initAttr->npuNum = 128; // Total number of NPUs involved in monitoring, that is, the number of NPUs to be initialized.
    initAttr->bufferSize = 4096 * 10; // The configuration value must be 4 KB aligned and must be greater than Number of packets sent to each target × 2048 × Number of target NPUs.
    initAttr->ipAddr = new char[ipLen_];
    strcpy(initAttr->eid, device_eid[devId]);
    HccnRpingCtx rpingCtx = nullptr;
    aclrtSetDevice(devId); // Specify a device before initialization.
    HccnRpingInit(devId, initAttr, &rpingCtx); // Call the initialization API to obtain RPingCtx.
    • The sample code for Atlas A3 training product/Atlas A3 inference product and Atlas A2 training product/Atlas A2 inference product is as follows:
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
          // Configure initialization information.
    int ipLen = 16;
    int devId = 4;
    HccnRpingInitAttr *initAttr = new HccnRpingInitAttr();
    initAttr->mode = HCCN_RPING_MODE_ROCE; // Set the network type to RoCE.
    initAttr->port = 13886; // Listening port, which can be an unused port.
    initAttr->npuNum = 128; // Total number of NPUs involved in monitoring, that is, the number of NPUs to be initialized.
    initAttr->bufferSize = 4096 * 50; // The configuration value must be 4 KB aligned and must be greater than Number of packets sent to each target × 2048 × Number of target NPUs.
    initAttr->ipAddr = new char[ipLen];
    strcpy(initAttr->ipAddr, deviceIp[devId]);
    HccnRpingCtx rpingCtx = nullptr;
    aclrtSetDevice(devId); // Specify a device before initialization.
    HccnRpingInit(devId, initAttr, &rpingCtx); // Call the initialization API to obtain RPingCtx.
  2. Add a ping target to establish RPing communication between the client NPU and the target NPU.
    Call the HccnRpingAddTarget API to add a target for the client NPU. The target is the NPU to be monitored.
    • Before calling this API, define the ping target using HccnRpingTargetInfo.
    • After this API is called, if the target is not initialized through HccnRpingInit within 120s, the initialization times out and fails.
    • This API can be called repeatedly to implement target deduplication based on the IP address. Only the first target is retained.
    • If a target fails to be added, an error is returned.
    • A maximum of 16 targets can be added at once. If more than 16 targets need to be added, call this API for multiple times.
    The following is the sample code for adding a ping target:
    • Atlas 350 Accelerator Card
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
          // Add a ping target.
    char device_eid[8][33] = {
        "000000000000002000100000dfdf1701",
        "000000000000002000100000dfdf1702",
        "000000000000002000100000dfdf1703",
        "000000000000002000100000dfdf1704",
        "000000000000002000100000dfdf1705",
        "000000000000002000100000dfdf1706",
        "000000000000002000100000dfdf1707",
        "000000000000002000100000dfdf1708"
    };
    int ipLen = 32;

    int targetNum = devices.size() - 1;
    HccnRpingTargetInfo *target = new HccnRpingTargetInfo[targetNum];
    for (int i = 0; i < devices.size() - 1; i++) {
        int devTargetId = devices[i]; // Obtain the target deviceLogicId from the devices vector.
        target[i].srcPort = 0; // TCP/UDP source port.
        target[i].sl = 0; // Service level of the RDMA NIC. Value range: [0, 7].
        target[i].tc = 0; // Traffic class of the RDMA NIC. Set it to DSCP value of RoCE packets × 4.
        target[i].port = 13886; // Socket connection port number, which is the same as the port number of the target device during initialization.
        char payload[12] = "hellotarget";
        strcpy(target[i].payload, payload);
        target[i].addrType = 1;
        target[i].srcEid = new char[ipLen_];
        target[i].dstEid = new char[ipLen_];
        strcpy(target[i].srcEid, device_eid[devClientId]);
        strcpy(target[i].dstEid, device_eid[devTargetId]);
    }
    if (rpingCtx == nullptr) {
        printf("rpingCtx is null!\n");
        return -1;
    }
    HccnRpingAddTarget(rpingCtx, targetNum, target);

      The HccnRpingAddTarget API can deduplicate the target array in the input based on the value of dstEid in the HccnRpingTargetInfo object. The value of targetNum must be the same as the target array size. Otherwise, out-of-bounds array access may occur.

      To customize the RPing timeout interval, use the HccnRpingAddTargetWithCfg API.

    • Atlas A3 training product/Atlas A3 inference product and Atlas A2 training product/Atlas A2 inference product
       1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
          // Add a ping target.
    char deviceIp[8][16] = 
    {   
        "192.168.99.127",
        "192.168.99.128",
        "192.168.99.129",
        "192.168.99.130",
        "192.168.99.131",
        "192.168.99.132",
        "192.168.99.133",
        "192.168.99.134"
    };
    int ipLen = 16;

    int targetNum = devices.size() - 1;
    HccnRpingTargetInfo *target = new HccnRpingTargetInfo[targetNum];
    for (int i = 0; i < devices.size() - 1; i++) {
        int devTargetId = devices[i]; // Obtain the target deviceLogicId from the devices vector.
        target[i].srcPort = 0; // TCP/UDP source port.
        target[i].sl = 4; // Service level of the RDMA NIC. Value range: [0, 7].
        target[i].tc = (33 & 0x3f) << 2; // Traffic class of the RDMA NIC. Set it to DSCP value of RoCE packets × 4.
        target[i].port = 13886; // Socket connection port number, which is the same as the port number of the target device during initialization.
        target[i].srcIp = new char[ipLen];
        target[i].dstIp = new char[ipLen];
        strcpy(target[i].srcIp, deviceIp[devClientId]);
        strcpy(target[i].dstIp, deviceIp[devTargetId]);
    }
    if (rpingCtx == nullptr) {
        printf("rpingCtx is null!\n");
        return -1;
    }
    HccnRpingAddTarget(rpingCtx, targetNum, target); 
    /* 
    The HccnRpingAddTargetWithCfg API can be used to customize the RPing timeout interval. The following is an example:
    HccnRpingAddTargetConfig config;
    config.connectTimeout = 120; // RPing timeout interval. The value range is [1,3600000], in ms.
    HccnRpingAddTargetWithCfg(rpingCtx, targetNum, target, &config);
    */

      The HccnRpingAddTarget API can deduplicate the target array in the input based on the value of dstIp in the HccnRpingTargetInfo object. The value of targetNum must be the same as the target array size. Otherwise, out-of-bounds array access may occur.

      To customize the RPing timeout interval, use the HccnRpingAddTargetWithCfg API.

  3. Start a ping request.
    Call HccnRpingBatchPingStart to initiate a ping request and start the BatchPing task. The client NPU sends ping packets to the target NPU based on the number of packets to be sent and interval time configured in this API.
    • Before calling this API, ensure that the target NPU to be monitored has been added to the client NPU and the ping task (BatchPing task) is not being executed.
    • To call this API repeatedly, you can first call HccnRpingBatchPingStop to stop the BatchPing task.

    The following is a code snippet for starting a ping request:

    1
2
3
4
        uint32_t pktNum = 10;   // Number of packets sent to each target.
    uint32_t interval = 1;  // Packet TX interval, in milliseconds.
    uint32_t timeout = 100; // Maximum timeout from packet TX to packet RX, in milliseconds.
    HccnRpingBatchPingStart(rpingCtx, pktNum, interval, timeout);

    Before the HccnRpingBatchPingStart API is called, bufferSize configured in HccnRpingInit should be verified. The bufferSize value must be greater than pktNum × 2048 × targetNum, where pktNum is the number of packets sent to each target, and targetNum is the total number of target NPUs to be monitored by the client NPU.

  4. Call HccnRpingGetResult to obtain the ping result.

    The ping result is displayed as an array. Each array element stores the information returned by a ping target. This API does not support deduplication. Therefore, ensure that the target argument does not contain duplicate information. The state field indicates that the ping result is valid. Each target result has a corresponding state. The value HCCN_RPING_RESULT_STATE_VALID indicates that the result is valid. Otherwise, the result is invalid.

    The following is a code snippet for obtaining the ping result:

     1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
        // Obtain the ping result.
    HccnRpingResultInfo *result = new HccnRpingResultInfo[targetNum];
    HccnResult hccnRet = HCCN_E_AGAIN;
    while(hccnRet == HCCN_E_AGAIN) {
        sleep(1);
        hccnRet =  HccnRpingGetResult(rpingCtx, targetNum, target, result);
    }
    for (int i = 0; i < targetNum; i++) {
        printf("txPkt[%u] rxPkt[%u] minRTT[%u] maxRTT[%u] avgRTT[%u] state[%u] payloadLen[%u]\n",
            result[i].txPkt, // Number of TX packets.
            result[i].rxPkt, // Number of RX packets.
            result[i].minRTT,// Minimum RTT of RX and TX packets.
            result[i].maxRTT,// Maximum RTT of RX and TX packets.
            result[i].avgRTT,// Average RTT of RX and TX packets.
            result[i].state);// Obtain the result state.
    }

    unsigned int payloadLenOutput = 0;
    void* payloadOutput;
    HccnRpingGetPayload(rpingCtx, &payloadOutput, &payloadLenOutput);
    int payloadNum = payloadLenOutput / 2048;
    for (int i = 0; i < payloadNum; i++) {
        HccnRpingPayloadHead *head = static_cast<HccnRpingPayloadHead*>(payloadOutput);
        printf("[%dth] srcIp:%s, dstIp:%s, payloadLen:%d, t1:%llu %llu, t2:%llu %llu, t3:%llu %llu, t4:%llu %llu, task id:%u\n",
			i,
                        head->srcIp, head->dstIp,
                        head->payloadLen,
			head->t1.sec, head->t1.usec,
			head->t2.sec, head->t2.usec,
			head->t3.sec, head->t3.usec,
			head->t4.sec, head->t4.usec,
			head->rpingBatchId);
        char* ptrTmp = static_cast<char*>(payloadOutput);
	ptrTmp += 2048;
	payloadOutput = ptrTmp;
    }
  5. Stop the ping task.

    After obtaining the result, call HccnRpingBatchPingStop to stop the ping task.

    1
2
        // Stop the ping task.
    HccnRpingBatchPingStop(rpingCtx);
  6. Remove the ping target.

    Before the ping task is stopped, remove the added ping target, and disconnect the established RPing communication.

    • Before calling this API, ensure that no ping task is being executed.
    • Before calling this API, ensure that the target has been added.
    1
2
        // Remove the ping target.
    HccnRpingRemoveTarget(rpingCtx, targetNum, target);
  7. Deinitialize RPing resources.

    Before the end of the process, call HccnRpingDeinit to deinitialize memory resources.

    • This API can be called only after HccnRpingInit is called successfully. Otherwise, nonexistent memory resources will be deinitialized, causing exceptions.
    • This API can be called only once. Otherwise, memory resources will be deinitialized repeatedly, which may cause segmentation fault or even coredump.
    1
2
        // Deinitialize memory.
    HccnRpingDeinit(rpingCtx);
This site uses cookies to improve your browsing experience. By continuing to browse this site, you agree to our use of cookies.View Details