LLM-DataDist Constructor

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference products	x
Atlas training products	x

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server, Atlas 300I A2 inference card, and A200I A2 Box heterogeneous components are supported.

Function Description

Constructs an LLM-DataDist instance.

Prototype

__init__(role: LLMRole, cluster_id: int)

Parameters

Parameter	Data Type	Description
role	LLMRole	Cluster role. This parameter is used only to identify the current role and does not affect the transmission process. Options: LLMRole.DECODER: incremental clusters LLMRole.PROMPT: full clusters
cluster_id	int	Cluster ID that identifies the LLM-DataDist instance. It must be unique among all participants in the link establishment.

Parameter

Data Type

Description

role

LLMRole

Cluster role. This parameter is used only to identify the current role and does not affect the transmission process. Options:

LLMRole.DECODER: incremental clusters
LLMRole.PROMPT: full clusters

cluster_id

int

Cluster ID that identifies the LLM-DataDist instance. It must be unique among all participants in the link establishment.

Example

from llm_datadist import LLMDataDist, LLMRole
llm_datadist = LLMDataDist(LLMRole.DECODER, 0)

Returns

In normal cases, an LLM-DataDist instance is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Restrictions

None

Parent topic: LLMDataDist