set_split_strategy_by_idx

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	☓
Atlas inference product	√
Atlas training product	√

For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800T A2 training server, Atlas 900 A2 PoD cluster basic unit, and Atlas 200T A2 Box16 heterogeneous subrack are supported.

For the Atlas inference products, only the Atlas 300I Duo inference card is supported.

Function

Sets a gradient segmentation strategy in a collective communication group based on the gradient index ID to implement AllReduce fusion and optimize the collective communication performance.

Prototype

def set_split_strategy_by_idx(idxList, group="hccl_world_group")

Parameters

Parameter	Input/Output	Description
idxList	Input	A list. Index ID list of gradients. The index ID list of the gradient must be a non-negative number in ascending order. The gradient index ID must be set based on the total number of gradient parameters of the model. The index ID starts from 0. The maximum value can be obtained as follows: Do not call the gradient segmentation API to set the gradient segmentation policy for training. In this case, the script uses the default gradient segmentation mode in set_split_strategy_by_size for training. After training, search for the keyword segment result in the INFO-level training logs on the host to obtain the gradient segmentation details, for example, segment index list: [0, 107] [108, 159]. The maximum number (for example, 159) in the segment sequence is the maximum value of the total gradient parameter index. NOTE: During the training process, logs may be overwritten. In this case, you can modify LogAgentMaxFileNum in /var/log/npu/conf/slog/slog.conf to increase the number of log files that can be stored on the host. Alternatively, you can also perform only one iteration. A maximum of eight gradient segments are supported. For example, if a model has 160 parameters that generate gradients and need to be divided into three segments: [0, 20], [21, 100], and [101, 159], set idxList to [20, 100, 159].
group	Input	A string. Group name, which can be a user-defined value or hccl_world_group. Defaults to hccl_world_group.

Parameter

Input/Output

Description

idxList

Input

A list.

Index ID list of gradients.

The index ID list of the gradient must be a non-negative number in ascending order.
The gradient index ID must be set based on the total number of gradient parameters of the model. The index ID starts from 0. The maximum value can be obtained as follows:
- Do not call the gradient segmentation API to set the gradient segmentation policy for training. In this case, the script uses the default gradient segmentation mode in set_split_strategy_by_size for training.
- After training, search for the keyword segment result in the INFO-level training logs on the host to obtain the gradient segmentation details, for example, segment index list: [0, 107] [108, 159]. The maximum number (for example, 159) in the segment sequence is the maximum value of the total gradient parameter index.
  NOTE:
  During the training process, logs may be overwritten. In this case, you can modify LogAgentMaxFileNum in /var/log/npu/conf/slog/slog.conf to increase the number of log files that can be stored on the host. Alternatively, you can also perform only one iteration.
A maximum of eight gradient segments are supported.
For example, if a model has 160 parameters that generate gradients and need to be divided into three segments: [0, 20], [21, 100], and [101, 159], set idxList to [20, 100, 159].

group

Input

A string.

Group name, which can be a user-defined value or hccl_world_group. Defaults to hccl_world_group.

Returns

None

Constraints

The caller rank must be within the range defined by the group argument passed to this API call. Otherwise, the API call fails.
If you do not call the gradient segmentation API to set the segmentation policy, the default backward gradient segmentation policy is used.
Default segmentation strategy: two segments with the first taking up 96.54% of the gradient data size, and the second segment taking up 3.46% (In some cases, there is only one segment).

Call Example

from hccl.split.api import *
set_split_strategy_by_idx([20, 100, 159], "group")

Parent topic: hccl.split.api