LLMDataDist Constructor

Applicable Products

Product	Supported (Yes/No)
Ascend 950PR/Ascend 950DT	No
Atlas A3 training products/Atlas A3 inference products	Yes
Atlas A2 training products/Atlas A2 inference products	Yes
Atlas 200I/500 A2 inference products	No
Atlas inference products	No
Atlas training products	No

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Constructs LLMDataDist.

Prototype

__init__(role: LLMRole, cluster_id: int)

Parameters

Parameter	Data Type	Value Description
role	LLMRole	Cluster role. The options are as follows: LLMRole.DECODER: incremental cluster, which can only be used as a client. LLMRole.PROMPT: full cluster, which can only be used as a server. LLMRole.MIX: mixed deployment.
cluster_id	int	Cluster ID that identifies the LLM-DataDist instance. It must be unique among all participants in the link establishment.

Parameter

Data Type

Value Description

role

LLMRole

Cluster role. The options are as follows:

LLMRole.DECODER: incremental cluster, which can only be used as a client.
LLMRole.PROMPT: full cluster, which can only be used as a server.
LLMRole.MIX: mixed deployment.

cluster_id

int

Cluster ID that identifies the LLM-DataDist instance. It must be unique among all participants in the link establishment.

Example

from llm_datadist import LLMDataDist, LLMRole
llm_datadist = LLMDataDist(LLMRole.DECODER, 0)

Returns

In normal cases, the LLMDataDist instance is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Constraints

None

Parent topic: LLMDataDist