LLMDataDist Constructor

Applicability

Product	Supported (√/x)
Atlas 350 Accelerator Card	x
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Note: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Constructs LLMDataDist.

Prototype

__init__(role: LLMRole, cluster_id: int)

Parameters

Parameter	Data Type	Description
role	LLMRole	Cluster role. This parameter is used only to identify the current role and does not affect the transmission process. The options are as follows: LLMRole.DECODER: incremental clusters LLMRole.PROMPT: full clusters
cluster_id	int	Cluster ID that identifies the LLM-DataDist instance. It must be unique among all participants in the link establishment.

Parameter

Data Type

Description

role

LLMRole

Cluster role. This parameter is used only to identify the current role and does not affect the transmission process. The options are as follows:

LLMRole.DECODER: incremental clusters
LLMRole.PROMPT: full clusters

cluster_id

int

Cluster ID that identifies the LLM-DataDist instance. It must be unique among all participants in the link establishment.

Example

from llm_datadist import LLMDataDist, LLMRole
llm_datadist = LLMDataDist(LLMRole.DECODER, 0)

Returns

In normal cases, the LLMDataDist instance is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Restrictions

None

Parent topic: LLMDataDist