Description of Command-Line Options

-f <hostfile>

Optional

Hostfile node list file.

Configure this file in multi-server scenarios. It can be configured as the absolute path of the Hostfile, or a relative path relative to the current command execution directory.

For details about the configuration example, see 4.

-n <number>

Required

Total number of NPUs to be started, that is, Number of nodes × Number of NPUs participating in training on each node.

--prefix <mpi_install_path>

Optional

Installation path of Open MPI.

Generally, this parameter does not need to be configured in single-server scenarios, but must be configured in multi-server scenarios. Otherwise, issues such as failure to obtain MPI library files may occur.

-hostfile <hostfile>

Optional

Hostfile node list file.

Configure this file in multi-server scenarios. It can be configured as the absolute path of the Hostfile, or a relative path relative to the current command execution directory.

For details about the configuration example, see 4.

-n <number>

Required

Total number of NPUs to be started, that is, Number of nodes × Number of NPUs participating in training on each node.

-x <env>

Required

Name of the environment variable to be passed to the remote node.

--allow-run-as-root

Optional

The mpirun command can be run by root users.

--mca <key value>

Optional

The Open MPI is centered on MPI Component Architecture (MCA). You can set mca at mpirun runtime to load various Open MPI components to implement certain features.

Common commands:

• --mca btl_tcp_if_include <nic_name>

  Use a specified NIC for inter-node communication. For example:

  --mca btl_tcp_if_include eth0

• --mca opal_set_max_sys_limits 1

  Configures the Open MPI runtime to inherit the system ulimit settings (such as the number of file descriptors), ensuring that Open MPI processes execute without encountering resource limit constraints. When deploying a cluster with a large number of NPUs, enabling this configuration is recommended to prevent system resource exhaustion.

./bin/<executable_file>

Required

Command of the HCCL Performance Tester.

<executable_file> is the executable file of the HCCL Performance Tester containing supported test commands.

• For Atlas 350 Accelerator Card, the supported test commands are all_gather_test, all_gatherv_test, all_reduce_test, alltoall_test, alltoallv_test, alltoallvc_test, broadcast_test, reduce_scatter_test, reduce_scatterv_test, reduce_test, and scatter_test.
• For Atlas A3 training product/Atlas A3 inference product, the supported test commands are all_gather_test, all_gatherv_test, all_reduce_test, alltoall_test, alltoallv_test, alltoallvc_test, broadcast_test, reduce_scatter_test, reduce_scatterv_test, reduce_test, and scatter_test.
• For Atlas A2 training product/Atlas A2 inference product, the supported test commands are all_gather_test, all_gatherv_test, all_reduce_test, alltoall_test, alltoallv_test, alltoallvc_test, broadcast_test, reduce_scatter_test, reduce_scatterv_test, reduce_test, and scatter_test.
• For Atlas training product, the supported test commands are all_gather_test, all_reduce_test, alltoallv_test, alltoall_test, broadcast_test, reduce_scatter_test, reduce_test, and scatter_test.
• For Atlas inference product, the supported test commands are all_gather_test, all_gatherv_test, all_reduce_test, alltoall_test, alltoallv_test, reduce_scatter_test, and reduce_scatterv_test.

Options supported by the collective communication performance test

-p <npus>

or --npus <npus>

Optional

Number of NPUs participating in training on a single compute node.

The default value is the total number of NPUs on the current node. If the number of NPUs participating in training on a single compute node is less than the total number of NPUs on the current node, this parameter is mandatory.

Note: The HCCL Performance Tester starts the corresponding devices based on the configured number of NPUs used in training. For details about the restrictions of this option, see Restrictions.

-b <minbytes>

or --minbytes <minbytes>

Optional

Examples:

• Configuration example: -b 100M -e 400M -i 0

  This indicates that the test is performed continuously based on the 100 MB initial value of the test data size.

• Configuration example: -b 100M -e 400M -i 500

  This indicates that the test is performed using 100 MB as the initial value, with the data size increasing by an incremental step of 500 bytes each time, until the test ends.

• Configuration example: -b 100M -e 400M -f 2

  This indicates that the initial value of the test data size is 100 MB, the end value is 400 MB, and the data increment multiplication factor is 2. In this case, data sizes of 100 MB, 200 MB, and 400 MB will be used for testing in each respective iteration.

-e <maxbytes>

or --maxbytes <maxbytes>

Optional

-i <incsize>

or --stepbytes <incsize>

Optional

-f <incfactor>

or --stepfactor <incfactor>

Optional

-o <operator>

or --op <operator>

Optional

Operation type of the Reduce command. The value can be sum, prod, max, or min. The default value is sum.

Reduce-related commands include all_reduce_test, reduce_scatter_test, reduce_scatterv_test, and reduce_test.

• For Atlas 350 Accelerator Card, the supported operation types for all Reduce-related commands are sum, max, and min. prod is not supported.
• For the reduce_scatterv_test command:
  • Atlas 350 Accelerator Card: The supported operation types are sum, max, and min.
  • Atlas A3 training product/Atlas A3 inference product: The supported operation types are sum, max, and min.
  • Atlas A2 training product/Atlas A2 inference product: The supported operation types are sum, max, and min.
  • Atlas inference product: Only the sum operation type is supported.

-r <root>

or --root <root>

Optional

When the broadcast_test, reduce_test, or scatter_test command is executed, you can use this option to specify the device ID of the root node.

Value range: [0, Actual number of devices – 1]

Default value: 0

-d <datatype>

or --datatype <datatype>

Optional

Data type supported by the HCCL command. The default type is fp32.

• For the all_reduce_test, reduce_scatter_test and reduce_test commands:
  • Atlas 350 Accelerator Card: The supported data types are int8, int16, int32, int64, uint64, fp16, fp32, fp64, and bfp16.
  • Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, int16, int32, int64, fp16, fp32, and bfp16. The prod operation does not support the int16 and bfp16 data types.
  • Atlas A2 training product/Atlas A2 inference product: The supported data types are int8, int16, int32, int64, fp16, fp32, and bfp16. The prod operation does not support the int16 and bfp16 data types.
  • Atlas training product: The supported data types are int8, int32, int64, fp16, and fp32.
  • Atlas inference product: The supported data types are int8, int16, int32, fp16, and fp32. The prod, max, and min operations do not support the int16 data type.
• For the broadcast_test, all_gather_test, alltoallv_test, alltoallvc_test, alltoall_test, scatter_test, and all_gatherv_test commands, the supported data types are supported: int8, uint8, int16, uint16, int32, uint32, int64, uint64, fp16, fp32, fp64, bfp16, fp8e5m2, fp8e4m3, fp8e8m0, hif8.

  Note that:

  The bfp16 data type is supported only by the following Products:

  Atlas 350 Accelerator Card

  Atlas A3 training product/Atlas A3 inference product

  Atlas A2 training products

  The fp8e5m2, fp8e4m3, fp8e8m0, and hif8 data types are supported only by the following Products:

  Atlas 350 Accelerator Card

• For the reduce_scatterv_test command:
  • Atlas 350 Accelerator Card: The supported data types are int8, int16, int32, fp16, fp32, and bfp16.
  • Atlas A3 training product/Atlas A3 inference product: The supported data types are int8, int16, int32, fp16, fp32, and bfp16.
  • Atlas A2 training product/Atlas A2 inference product: The supported data types are int8, int16, int32, fp16, fp32, and bfp16.
  • Atlas inference product: The supported data types are int16, fp16, and fp32.

-z <0/1>

or --zero_copy <0/1>

Optional

Specifies whether to enable the zero copy function.

In single-operator mode, the input and output buffers of the operator dynamically change, so HCCL uses a transit buffer to facilitate collective communication, introducing additional memory copy overhead. The zero copy function reduces this memory copy overhead by directly operating on the memory passed by the service, thereby improving performance.

Note: The zero copy function is for trial use and may be changed in later versions. It is not currently supported for use in commercial products.

The zero copy function takes effect under the following conditions:

• Supported only on the Atlas A3 training product/Atlas A3 inference product.
• Supported only when executing the reduce_scatter_test, all_gather_test, all_reduce_test, and broadcast_test commands.
• Supported only when the communication operator expansion mode is AI_CPU.

  For details about the communication operator expansion mode, see the HCCL_OP_EXPANSION_MODE environment variable in the Environment Variables documentation.

-m <0/1>

or --symmetric_memory <0/1>

Optional

Specifies whether to enable the symmetric memory function.

In single-operator mode, the input and output buffers of the operator dynamically change, so HCCL uses a transit buffer to facilitate collective communication, introducing additional memory copy overhead. The symmetric memory function reduces this memory copy overhead by directly operating on the memory passed by the service, thereby improving performance.

This parameter supports the following values:

• 0 (default): Disables the symmetric memory function.
• 1: Enables the symmetric memory function.

The symmetric memory function takes effect under the following conditions:

• Supported only on the Atlas A3 training product/Atlas A3 inference product.
• Supported only for intra-SuperPoD communication.
• Supported only when executing the reduce_scatter_test, all_gather_test, all_reduce_test, and alltoall_test commands.
• Supported only when the communication operator expansion mode is AI_CPU.

  For details about the communication operator expansion mode, see the HCCL_OP_EXPANSION_MODE environment variable in the Environment Variables documentation.

• Supported only in scenarios where SDMA communication is performed between AI nodes in a SuperPoD using HCCS links. Scenarios using RoCE for RDMA communication are not supported. That is, the environment variable HCCL_INTER_HCCS_DISABLE cannot be set to TRUE.
• Supported only in symmetric networking scenarios where each server has the same number of cards.

Note: The zero copy and symmetric memory functions cannot be enabled at the same time.

-a <HcclAccelerator>

or --accelerator <HcclAccelerator>

Optional

This parameter is supported only for Atlas 350 Accelerator Card and is used to configure the acceleration mode.

• default: Uses the default adaptive acceleration mode, which automatically selects the optimal mode based on networking, data volume, and other conditions.
• aicpu_ts: Uses the AICPU compute units on the device side for acceleration.
• aiv: Uses the Vector Core compute units on the device side for acceleration. This configuration is not supported in the current version.
• ccu_ms: Uses the CCU memory slice (MS) mode for acceleration. This configuration is not supported in the current version.
• ccu_sched: Uses the CCU scheduling mode for acceleration.
• host_ts: This configuration is not supported.
• aiv_only: This configuration is not supported.

Note: This configuration takes precedence over the HCCL_OP_EXPANSION_MODE environment variable.

-s <0/1>

or --nslb <0/1>

Optional

Performance test options

-n <iters_count>

or --iters <iters_count>

Optional

Number of iterations. The default value is 20.

-w <warmup_iters_count>

or --warmup_iters <warmup_iters_count>

Optional

Number of warm-up iterations. This option affects only the execution duration of the HCCL Performance Tester, and is not counted in performance statistics. The default value is 10.

Note: Due to the possibility of operations in the first few iterations that may affect the performance test, such as socket establishment operations in the first iteration, you are advised to set the first few iterations as warm-up iterations and not include them in performance statistics.

-t or --onlydevicetim

Optional

Result check options

-c <0/1>

or --check <0/1>

Optional

Note: In large-scale cluster scenarios, enabling result check will increase the execution duration of the HCCL Performance Tester.