文档
下载
文档中心
Basic Functions

Basic Functions

CLUSTER_CONFIG

Sets the path of the cluster configuration file, which is used to configure the target execution logic device information for generating HCCL tasks in multi-card distributed training scenarios. The actual corresponding options parameter is ge.cluster_config.

If the model contains communication operators (currently only the AllGather communication operator is supported), this parameter is mandatory

Configuration example:

{ge::ir_option::CLUSTER_CONFIG, "/home/test/cluster_config.json"}

The following is a configuration example of cluster_config.json:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
{
    "RankTable": {
        "status": "completed",
        "version": "1.2",
        "server_count": "2",
        "server_list": [
            {
                "server_id": "node_0",
                "device": [
                    {
                        "device_id": "0",
                        "rank_id": "0"
                    }
                ]
            },
            {
                "server_id": "node_1",
                "device": [
                    {
                        "device_id": "0",
                        "rank_id": "1"
                    }
                ]
            }
        ]
    },
    "HcclCommConfig": {
        "hcclCommName": "global name"
    }
}

The parameters are described as follows:

  • RankTable: core of the configuration, which defines the nodes and ranks involved in computation.
    • status: (mandatory) indicator of whether the rank table is available.
      • completed: The rank table is available.
      • initializing: The rank table is unavailable.
    • version: (mandatory) version of the rank table template. Set this parameter to 1.2.
    • server_count: (optional) number of AI Servers participating in collective communication.
    • server_list: (mandatory) list of AI Servers participating in collective communication.
      • server_id: (mandatory) AI Server ID. The value is a string of up to 64 characters, for example, node_0. Ensure that the value is globally unique.
      • device: (mandatory) device list.
        • device_id: (mandatory) physical ID of the AI processor, that is, the serial number of the device on the AI Server.
        • rank_id: (mandatory) unique ID of a rank. The value must be an integer starting from 0 and must be globally unique. Value range: [0, Total number of devices – 1].
  • HcclCommConfig: global configuration of the HCCL communication layer.
    • hcclCommName: (mandatory) name or identifier of a global communication group.

Applicable Products:

  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas 200/300/500 inference product: not supported

DETERMINISTIC

Whether to enable deterministic computing. The corresponding options parameter is ge.deterministic.

By default, deterministic computing is disabled. Running an operator multiple times with the same hardware and input may generate different results. This is generally caused by asynchronous multi-thread executions during operator implementation, which changes the accumulation sequence of floating-point numbers. When deterministic computing is enabled, the same output is generated if an operator is executed for multiple times with the same hardware and input.

You are advised not to enable deterministic computing because it slows down operator execution and affects performance. If the results of a model vary across multiple executions or the precision needs to be tuned, you can enable deterministic computing to assist model debugging and optimization.

Arguments:

  • 0 (default): Disables deterministic computing.
  • 1: Enables deterministic computing.

Configuration example:

{ge::ir_option::DETERMINISTIC, "1"}

Applicable products:

  • Atlas 200/300/500 inference product: supported
  • Atlas inference product: supported
  • Atlas 350 Accelerator Card: supported
  • Atlas 200/300/500 inference product: not supported
  • Atlas 200/300/500 inference product: supported
  • Atlas inference product: supported

ENABLE_SINGLE_STREAM

Specifies whether to enable single-stream serial execution of model inference in the static shape scenario. The corresponding options parameter is ge.enableSingleStream.

Streams preserve the order of a stack of asynchronous operations being executed on the device.

Arguments:

  • true: Enables in-order single-stream serial execution of model inference.
  • false (default): Disables in-order single-stream serial execution of model inference.

Restrictions:

The single-stream feature is not available if the model contains Cmo operators and control operators below. In this case, use the default value false.

  • Merge
  • Switch
  • Enter
  • RefEnter

Configuration example:

{ge::ir_option::ENABLE_SINGLE_STREAM, "true"}

Applicable products:

  • Atlas 200/300/500 inference product: supported
  • Atlas inference product: supported
  • Atlas 350 Accelerator Card: supported
  • Atlas 200/300/500 inference product: not supported
  • Atlas 200/300/500 inference product: supported
  • Atlas 200/300/500 inference product: supported

HCCL_SUB_COMM_CONFIG

Sets the path of the HCCL sub-communication domain configuration file, which is used to configure HCCL sub-communication parameters in multi-card distributed training scenarios. The corresponding options parameter is ge.hccl_sub_comm_config.

If the model contains communication operators (currently only the AllGather communication operator is supported), this parameter is mandatory If the model operators include sub-communication domain parameters but HCCL_SUB_COMM_CONFIG is not specified during model compilation, the system will default to executing based on the actual cluster service configuration.

Configuration example:

{ge::ir_option::CLUSTER_CONFIG, "/home/test/cluster_config.json"}
{ge::ir_option::HCCL_SUB_COMM_CONFIG, "/home/test/sub_comm_config.json"}

The following is an example of the sub_comm_config.json file:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
{
  "GroupList": [
    {
      "RankIds": [0,1],
      "HcclCommconfig": {
        "hcclCommName": "test_group"
      }
    }
  ]
}

Notes:

  • RankIds: unique ID of a rank. The value ranges from 0 to number of devices – 1.

    You are advised to sort rank_id based on the physical connection sequence of devices, that is, arrange devices by physical connection proximity. Otherwise, the performance may be affected.

    For example, if device_ip is set in ascending order based on physical connections, you are advised to set rank_id in ascending order.

  • hcclCommName: name of a communication group, which is invoked and distinguished by programs.

Applicable products:

  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas inference product: not supported
  • Atlas 200/300/500 inference product: not supported

OPTION_HOST_ENV_OS

The corresponding options parameter is ge.host_env_os.

If the OS and its architecture of the model build environment are inconsistent with those of the model operating environment, set this parameter to the OS type of the model operating environment. If this parameter is not set, the OS type of the model build environment is used by default.

If the OS and its architecture of the model build environment are inconsistent with those of the model operating environment, use this option together with OPTION_HOST_ENV_CPU. OPTION_HOST_ENV_OS is used to set the OS type, and OPTION_HOST_ENV_CPU is used to set the OS architecture.

Argument: OS type of the operator .so file packaged in the ${INSTALL_DIR}/opp/built-in/op_graph/lib/ directory.

Default value: value in the ${INSTALL_DIR}/opp/scene.info file.

Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

Configuration example:

{ge::OPTION_HOST_ENV_OS, "linux"},
{ge::OPTION_HOST_ENV_CPU, "x86_64"}

Applicable products:

All processors

OPTION_HOST_ENV_CPU

The corresponding options parameter is ge.host_env_cpu.

If the OS and its architecture of the model build environment are inconsistent with those of the model operating environment, set this parameter to the OS architecture of the model operating environment. If this parameter is not set, the OS architecture of the model build environment is used by default.

If the OS and its architecture of the model build environment are inconsistent with those of the model operating environment, use this option together with OPTION_HOST_ENV_OS. OPTION_HOST_ENV_OS is used to set the OS type, and OPTION_HOST_ENV_CPU is used to set the OS architecture.

Argument: OS or CPU type of the operator .so file packaged in the ${INSTALL_DIR}/opp/built-in/op_graph/lib/ directory.

Default value: value in the ${INSTALL_DIR}/opp/scene.info file.

Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.

Configuration example:

{ge::OPTION_HOST_ENV_OS, "linux"},
{ge::OPTION_HOST_ENV_CPU, "x86_64"}
  • If the generated offline model contains the OS type and architecture, for example, xxx_linux_x86_64.om, the model can run only on the Linux x86_64 OS.
  • If the generated offline model does not contain the OS type and architecture, for example, xxx.om, all OSs supported by the CANN package support the model.

Applicable products:

All processors

SOC_VERSION

AI processor model used during graph build. The corresponding options parameter is ge.socVersion.

  • This parameter is optional if the current environment has an AI processor.
  • This parameter is required if the current environment does not have an AI processor, such as a development environment.

Arguments:

To query <soc_version>:

  • For the following products: Run the npu-smi info command on the server where AI processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.

    Atlas A2 training products/Atlas A2 inference products

    Atlas 200I/500 A2 inference products

    Atlas inference products

    Atlas training products

  • For the Atlas A3 training products/Atlas A3 inference products: Run the npu-smi info -t board -i id -c chip_id command on the server where the AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234.
    • id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
    • chip_id: chip ID, which is obtained by running the npu-smi info -m command.
  • For the Ascend 950PR/Ascend 950DT: Run the npu-smi info -t board -i id command on the server where the AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234.

    id indicates the device ID, which is the NPU ID obtained by running the npu-smi info -l command.

Configuration example:

{ge::ir_option::SOC_VERSION, "<soc_version>"}

Applicable products:

All processors

VIRTUAL_TYPE

Whether an offline model can run on a virtual device generated by the Ascend virtual instance feature. The corresponding options parameter is ge.virtual_type.

If the computing power of a chip is too much for cloud users or small enterprises, the Ascend virtual instance feature can be applied to allocate a proper amount of computing power as needed by the users or small enterprises to suit their services.

A virtual device is a virtual acceleration resource allocated by a chip based on specified computing power.

Arguments:

  • 0 (default): The offline model does not run on the virtual device generated by the Ascend virtual instance feature.
  • 1: The offline model runs on virtual devices with different computing power.

Configuration example:

{ge::ir_option::VIRTUAL_TYPE, "1"}

Restrictions:

  1. If model conversion is performed with this parameter set to 1, the number of computing logical AI Cores of the generated offline model may be greater than the actual number of cores specified by aicore_num. The value is the least common multiple supported by aicore_num.

    For example, if the value range of aicore_num is {1,2,4,8}, the number of NPU blocks may be 8.

  2. If this parameter is set to 1 and the generated model contains the following operators, a single core is used by default. In this case, the inference performance of the generated model deteriorates.
    • DynamicRNN
    • PadV2D
    • SquareSumV2
    • DynamicRNNV2
    • DynamicRNNV3
    • DynamicGRUV

Applicable products:

  • Atlas 200/300/500 inference product: supported
  • Atlas inference product: supported
  • Atlas inference product: supported
  • Atlas inference product: supported
  • Atlas inference product: supported
  • Atlas 350 Accelerator Card: supported
我们使用cookie来确保您的高速浏览体验。继续浏览本站，即表示您同意我们使用cookie。查看详情