文档
下载
文档中心
Basic Functions

Basic Functions

ge.constLifecycle

Configures the lifecycle of constant nodes in the training and online inference scenario.

Parameter values:

  • session: Constant nodes are stored at the session level. If session is used, memory reuse is supported for constant nodes between multiple graphs in a session. However, ensure that constant nodes with the same name in multiple graphs are the same.
  • graph: Constant nodes are stored at the graph level. You can call SetGraphConstMemoryBase to manage the const memory at the graph level.

The default value is session in the training scenario and graph in the online inference scenario.

Configuration example:

{"ge.constLifecycle", "graph"};

Mandatory/Optional: optional

Effective level: global, session, and graph

ge.deterministic

Sets whether to enable deterministic computing.

By default, deterministic computing is disabled. Multiple execution results of an operator with the same hardware and input may be different. This is generally caused by asynchronous multi-thread executions during operator implementation, which changes the accumulation sequence of floating-point numbers. When deterministic computing is enabled, the same output is generated if an operator is executed for multiple times with the same hardware and input. This often slows down operator execution. If the execution results of a model are different for multiple times or the precision needs to be optimized, you can enable deterministic computing to assist model debugging and optimization.

Parameter values:

  • 0 (default): Disables deterministic computing.
  • 1: Enables deterministic computing.

Configuration example:

{"ge.deterministic", "0"};

Mandatory/Optional: optional

Effective level: global

ge.enableSingleStream

Sets whether to enable single-stream serial execution of graphs in the static shape scenario.

Streams preserve the order of a stack of asynchronous operations being executed on the device.

Parameter values:

  • true: Single-stream serial execution of graphs is enabled.
  • false (default): Multiple streams are executed concurrently during graph running.

Configuration example:

{"ge.enableSingleStream", "false"};

Mandatory/Optional: optional

Effective level: graph

Restrictions:

If the model contains the Cmo operator and the following control operators, the single-stream feature cannot be used. In this case, use the default value false.

  • Merge
  • Switch
  • Enter
  • RefEnter

ge.exec.deviceId

Logical ID of the operated device when the GE instance is running.

Parameter values:

  • In the online inference scenario, the value ranges from –1 to N – 1. The default value is -1.
  • In the training scenario, the value ranges from 0 to N – 1. The default value is 0.

N indicates the number of available AI processors on the server.

Configuration example:

{"ge.exec.deviceId", "-1"};

Mandatory/Optional: optional

Effective level: global

ge.exec.frozenInputIndexes

Sets the index of the input tensor whose address is not updated. This parameter can be called only for LoadGraph. The input tensor index varies according to the model.

  • Dynamic shape model: The index of the input tensor, address of the data on the device, and data length must be passed. The address must be in decimal format.
  • Static shape model: Only the index of the input tensor needs to be passed. Other parameters, such as data length, do not take effect.

Configuration example:

# Pass only the input tensor index.
{"ge.exec.frozenInputIndexes", "0;1;2"};
# Pass the input tensor index, address of the data on the device, and data length.
{"ge.exec.frozenInputIndexes", "0,88832131,4;1,888213294,4;2,193492421,2"};

For details about the usage examples and precautions, see Running a Graph Asynchronously in the Single-Process and Single-Device Mode.

Mandatory/Optional: optional

Effective level: graph

Restrictions:

The input tensor whose address is not refreshed must have a static shape. For a dynamic shape model, the input tensor must also have a static shape.

ge.exec.hostInputIndexes

Sets the input tensor index whose placement attribute is host in the in-line copy scenario. Multiple input tensor indexes are separated by semicolons (;).

In-line copy refers to the process of copying the input tensor data from host memory to device memory when the operator address of the model is updated.

Configuration example:

{"ge.exec.hostInputIndexes", "0;1;2"};

Mandatory/Optional: optional

Effective level: graph

Restrictions:

  • This parameter applies only to static shape models.
  • The in-line copy feature is not suitable for large data size input, as the performance may deteriorate.
  • If the model has only one input, this parameter cannot be used together with ge.exec.frozenInputIndexes. If the model has multiple inputs, this parameter cannot be used together with ge.exec.frozenInputIndexes to specify the same input.
  • After the parameter setting is loaded through the options configuration item, RunGraphAsync among subsequent execution APIs cannot be used.

ge.exec.rankTableFile

Describes the cluster information involved in collective communication, including the organization information about the server, device, and container. The value is a ranktable file path, including the file name.

Configuration example:

{"ge.exec.rankTableFile", "./yourfilepath/ranktable.json"};

Mandatory/Optional: optional

Effective level: global, session, and graph

ge.exec.rankId

Specifies the rank ID of the current process in a group. It is used for distributed training and collective communication, and identifies the current device in multi-device training. For a custom group, the rank starts from 0 in the group. For an HCCL world group, the rank ID is the same as the world rank ID. This parameter must be used in pair with ge.exec.rankTableFile.

Parameter values:

Integer in the form of a string:

  • World rank ID: indicates the rank ID of a process in an HCCL world group. The value ranges from 0 to (rank size – 1).
  • Local rank ID: indicates the rank ID of a process in a group on the server where the process is located. The value ranges from 0 to (local rank size – 1).

Configuration example:

{"ge.exec.rankId", "0"};

Mandatory/Optional: optional

Effective level: global, session, and graph

ge.graphRunMode

Sets the graph execution mode.

Parameter values:

  • 0 (default): online inference
  • 1: training

Configuration example:

{"ge.graphRunMode", "0"};

Mandatory/Optional: optional

Effective level: global and session

ge.outputDatatype

Sets the output data type of a graph.

Parameter values:

  • FP32
  • FP16
  • UINT8
  • INT8
  • UINT16
  • INT16
  • UINT32
  • INT32
  • UINT64
  • INT64
  • DOUBLE
  • HIF8: Only the Ascend 950PR / Ascend 950DT supports this data type.
  • FP8E5M2: Only the Ascend 950PR / Ascend 950DT supports this data type.
  • FP8E4M3FN: Only the Ascend 950PR / Ascend 950DT supports this data type.

After graph build, the preceding data types are displayed in the corresponding subgraph file as follows:

  • DT_FLOAT
  • DT_FLOAT16
  • DT_UINT8
  • DT_INT8
  • DT_UINT16
  • DT_INT16
  • DT_UINT32
  • DT_INT32
  • DT_UINT64
  • DT_INT64
  • DT_DOUBLE
  • DT_HIFLOAT8
  • DT_FLOAT8_E5M2
  • DT_FLOAT8_E4M3FN

Restrictions:

If no data type is specified, the data type of the node (operator) output at the last layer of the graph is used. If a data type is specified, the specified data type is used.

Configuration example:

{"ge.outputDatatype", "FP32"};

Mandatory/Optional: optional

Effective level: session and graph

ge.session_device_id

Specifies the logical ID of a device. Setting this parameter allows you to run different models on multiple devices by executing a single training script.

You can create multiple threads, each of which is a session. Each session transfers a different value of ge.session_device_id.

Configuration example:

{"ge.session_device_id", "0"};

Mandatory/Optional: optional

Effective level: session

ge.socVersion

Specifies the AI processor model for model build and optimization.

To query the value, perform the following:

  • For the following products: Run the npu-smi info command on the server where AI processor is installed to obtain the Name information. The actual value is AscendName. For example, if Name is xxxyy, the actual value is Ascendxxxyy.

    Atlas A2 training products / Atlas A2 inference products

    Atlas 200I/500 A2 inference products

    Atlas inference products

    Atlas training products

  • For the Atlas A3 training products / Atlas A3 inference products : Run the npu-smi info -t board -i id -c chip_id command on the server where the AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234.
    • id: device ID, which is the NPU ID obtained by running the npu-smi info -l command.
    • chip_id: chip ID, which is obtained by running the npu-smi info -m command.
  • For the Ascend 950PR / Ascend 950DT : Run the npu-smi info -t board -i id command on the server where the AI processor is installed to obtain the Chip Name and NPU Name information. The actual value is Chip Name_NPU Name. For example, if the value of Chip Name is Ascendxxx and the value of NPU Name is 1234, the actual value is Ascendxxx_1234.

    id indicates the device ID, which is the NPU ID obtained by running the npu-smi info -l command.

Mandatory/Optional: optional

Effective level: global, session, and graph

Parent topic: Options
我们使用cookie来确保您的高速浏览体验。继续浏览本站，即表示您同意我们使用cookie。查看详情