Latency Test

Function

The latency test is used to test the latency of CPU and NPU links.

Test Procedure

If you want to test the SuperPoD P2P latency using MPI, install and configure MPI in the environment in advance by referring to Installing and Configuring the MPI in CANN HCCL Performance Tester and configure the environment variables by referring to Compilation in CANN HCCL Performance Tester. Note that only Open MPI 4.1.5 is supported. Do not use parameters, configurations, and environment variables that are not mentioned in the description and examples. For details about other Open MPI execution parameters, environment variables, and configuration files, see Open MPI documentation.

Ensure that the entered IP address is valid.

Parameters

You can run either of the following commands to list the parameters of the latency test command:

ascend-dmi -l -h

ascend-dmi --latency --help

Table 1 lists only a test-specific parameter. For details about other common parameters, see Common Parameters.

Examples

• Latency test of the Atlas 900 A3 SuperPoD without parameters (omitting parameters defaults to querying the latency test results for device 0, displayed in both the h2d and d2h data flow directions)

  ascend-dmi -l -q

  If the following information is displayed, the tool is running properly.

  Device Id: 0
  Data Size: 512 B
  Device to Host Latency: 802.421 ns
  Host to Device Latency: 766.108 ns

• Latency test of the Atlas 900 A3 SuperPoD, with the data flow direction specified as h2d, the size set to 1024 bytes, and the device specified as 0

  ascend-dmi -l -t h2d -s 1024 -d 0 -q

  If the following information is displayed, the tool is running properly.

  Device Id: 0
  Data Size: 1024 B
  Host to Device Latency: 760.847 ns

• Latency test of the Atlas 900 A3 SuperPoD, with the data flow direction specified as h2d, the size set to 1024 bytes, the device specified as 0, and the output format set to JSON

  ascend-dmi -l -t h2d -s 1024 -d 0 --fmt json

  If the following information is displayed, the tool is running properly.

  {
      "Latency": [
          {
              "device_id": "0",
              "latency": "730.551392 ns",
              "size": "1024 Bytes",
              "type": "Host to Device Test"
          }
      ]
  }

• Intra-node P2P latency test in P2P mode of the Atlas 900 A3 SuperPoD, without specifying the source device ID and destination device ID

  ascend-dmi -l -t p2p -q

  If the following information is displayed, the tool is running properly.

  Peer to Peer Test Latency Matrix(ns)
  DataSize: 512B
     D\D         0         1         2         3         4         5         6         7         8         9        10        11        12        13        14        15
       0       ***    325.16    969.35    966.85    966.79    956.41    966.18    960.39    956.57    953.76    953.14    946.77    944.15    981.92    980.80    974.58
       1    330.05       ***    968.09    970.09    961.34    959.97    964.92    965.87    951.86    956.85    949.81    951.18    967.33    971.98    964.08    965.35
       2    939.32    968.42       ***    326.20    959.93    958.73    957.58    958.16    953.78    959.57    945.64    946.91    936.11    942.73    973.71    972.00
       3    958.91    957.44    340.43       ***    953.08    940.04    949.74    942.95    944.34    982.36    976.16    971.29    969.47    966.37    965.55    960.38
       4    963.15    962.73    954.09    957.77       ***    356.44    953.38    945.96    948.74    977.68    972.01    971.85    968.67    967.59    967.54    956.16
       5    950.61    942.05    939.59    967.77    324.42       ***    969.67    969.99    966.62    972.40    957.07    958.05    950.77    955.16    943.99    942.91
       6    948.66    948.03    968.72    969.73    967.50    965.83       ***    337.61    969.89    967.87    961.44    957.44    954.07    949.16    942.88    933.82
       7    976.93    975.45    967.34    968.76    959.31    957.11    349.05       ***    964.16    960.38    953.91    948.20    943.35    967.12    961.62    953.32
       8    963.13    952.56    953.37    949.14    950.89    970.35    981.88    980.45       ***    329.96    972.23    974.47    963.37    968.13    964.10    963.59
       9    964.94    953.57    954.26    951.08    952.50    972.18    982.47    981.44    330.59       ***    974.08    975.84    965.46    969.30    965.85    966.28
      10    964.13    963.64    951.72    954.40    948.32    945.63   1003.53    977.47    976.35    982.95       ***    335.24    969.35    966.56    965.56    960.00
      11    954.26    945.40    943.27    969.07    969.50    966.57    974.68    969.69    969.44    966.57    344.91       ***    955.75    960.55    949.62    948.76
      12    959.89    948.97    945.51    941.76    941.33    962.59    971.05    970.31    972.42    972.67    968.42    959.49       ***    347.29    952.97    952.85
      13    947.74    970.27    967.84    968.85    968.34    960.08    969.02    958.60    961.57    962.39    958.51    950.05    357.58       ***    941.05    940.14
      14    967.45    960.86    956.95    951.62    949.21    939.95    943.54    968.11    976.88    981.09    973.57    970.16    967.33    962.49       ***    346.76
      15    942.62    973.57    969.72    968.12    965.75    954.46    957.11    953.13    962.75    959.02    953.23    947.68    941.98    972.22    324.26       ***

• Intra-node P2P latency test in P2P mode of the Atlas 900 A3 SuperPoD, specifying the source device ID as 0 and destination device ID as 1

  ascend-dmi -l -t p2p --ds 0 --dd 1 -q

  If the following information is displayed, the tool is running properly.

  Device Id: 0 -> 1
  Data Size: 512 B
  Peer to Peer Latency: 360.144 ns

• SuperPoD P2P latency test using Open MPI
  

  • Parameter description:
    • -mca btl_tcp_if_include specifies the MPI communication standard and communication port.
    • -mca btl_tcp_port_min_v4 specifies the minimum TCP port number used by the MPI IPv4 standard.
    • -mca btl_tcp_port_min_v6 specifies the minimum TCP port number used by the MPI IPv6 standard.

  • No authentication mechanism is available for the ports used by MPI. To prevent malicious connection attacks, perform security hardening by referring to Avoiding MPI All-Zero Listening.

  Run the mpirun -mca btl_tcp_if_include xx.xx.xx.x2(Source node IP)/port -mca btl_tcp_port_min_v4 32768 -mca btl_tcp_port_min_v6 32768 -host xx.xx.xx.x2(Source node IP):1,xx.xx.xx.x3(Peer node IP):1 -n 2 ascend-dmi -l -t p2p --sp --ds 0 --dd 1 -q command to test the SuperPoD P2P latency.

  [****@node-97-52 xxx]# mpirun -mca btl_tcp_if_include xx.xx.xx.x2/22 -mca btl_tcp_port_min_v4 32768 -mca btl_tcp_port_min_v6 32768 -host xx.xx.xx.x2:1,xx.xx.xx.x3:1 -n 2 ascend-dmi -l -t p2p --sp --ds 0 --dd 1 -q
  Device Id: 0 -> 1
  Data Size: 512 B
  SuperPoD Peer to Peer Latency: 1726.06 ns

  Run the mpirun -mca btl_tcp_if_include xx.xx.xx.x2(Source node IP)/port -mca btl_tcp_port_min_v4 32768 -mca btl_tcp_port_min_v6 32768 -host xx.xx.xx.x2(Source node IP):1,xx.xx.xx.x3(Peer node IP):1 -n 2 ascend-dmi -l -t p2p --sp --ds 0 --dd 1 -q --fmt json command to test the SuperPoD P2P latency.

  [****@node-97-52 xxx]# mpirun -mca btl_tcp_if_include xx.xx.xx.x2/22 -mca btl_tcp_port_min_v4 32768 -mca btl_tcp_port_min_v6 32768 -host xx.xx.xx.x2:1,xx.xx.xx.x3:1 -n 2 ascend-dmi -l -t p2p --sp --ds 0 --dd 1 -q --fmt json
  {
      "Latency": [
          {
              "src_device_id": "0",
              "dst_device_id": "1",
              "latency": "1911.797119 ns",
              "size": "512 Bytes",
              "type": "SuperPod Peer to Peer Test"
          }
      ]
  }

• Intra-node P2P latency test in P2P mode of the Ascend 950PR and 950DT products, specifying the source device ID as 0 and destination device ID as 1

  ascend-dmi -l -t p2p --ds 0 --dd 1 -q

  If the following information is displayed, the tool is running properly.

  Device Id: 0 -> 1
  Data Size: 512 B
  Peer to Peer Latency: 800.144 ns

The table below describes the parameters in the preceding command output.