TransferConfig

Applicability

Product	Supported (√/x)
Atlas 350 Accelerator Card	x
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Note: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Constructs TransferConfig.

Prototype

__init__(dst_cluster_id: int, dst_addrs: List[int], src_layer_range: Optional[range] = None, src_batch_index: int = 0)

Parameters

Parameter	Data Type	Description
dst_cluster_id	int	Cluster ID of the instance where the destination cache is located.
dst_addrs	List[int]	Memory address of each tensor in the destination cache. If the destination cache is not in the PagedAttention scenario and batch_index for transmission is not 0, offset dst_addrs to the actual address.
src_layer_range	Optional[range]	Range of layers with data to be transmitted locally. The step must be 1. The default value is None, indicating that the data at all layers is transmitted.
src_batch_index	int	Batch index of the local cache. This parameter can be set when the source cache is in the non-PagedAttention scenario.

Example

from llm_datadist import TransferConfig
TransferConfig(1, dst_addrs, range(0, 3), 1)

Returns

In normal cases, a TransferConfig instance is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

If src_layer_range is invalid, an LLMException is thrown.

Restrictions

The number of addresses in the destination address list must be twice the number of layers with data to be transmitted.

Parent topic: API Reference