API List

LLM-DataDist is an LLM distributed cluster and data acceleration component. It provides clustered key-value (KV) data management capabilities and supports disaggregated deployment of full graphs and incremental graphs.
- LLM-DataDist APIs are stored in ${INSTALL_DIR}/include/llm_datadist/llm_datadist.h. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.
- The library file corresponding to LLM-DataDist APIs is libllm_datadist.so.
HIXL provides high-performance, zero-copy point-to-point data transmission capabilities for cluster scenarios and exposes its capabilities through simple APIs.
- HIXL APIs are stored in ${INSTALL_DIR}/include/hixl/hixl.h. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.
- The library file corresponding to HIXL APIs is libcann_hixl.so.

The following products are supported:

Atlas A2 training product/Atlas A2 inference product: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported. When the HCCS transmission protocol is used in the server, LLM-DataDist APIs support only D2D transmission.

Atlas A3 training product/Atlas A3 inference product: When the HCCS transmission protocol is used, LLM-DataDist APIs do not support the host memory as the remote cache.

The APIs are listed as follows.

**Table 1** LLM-DataDist APIs
API	Description
LlmDataDist Constructor	Constructs an LLM-DataDist object.
~LlmDataDist()	Destructs an LLM-DataDist object.
Initialize	Initializes the LLM-DataDist instance.
Finalize	Releases the LLM-DataDist.
SetRole	Sets the role of the current LLM-DataDist.
LinkLlmClusters	Establishes a link.
UnlinkLlmClusters	Disconnects a link.
PullKvCache	Pulls the KV cache in contiguous memory mode.
PullKvBlocks	Pulls the KV cache in block list mode.
PushKvCache	Pushes the cache to the remote node.
PushKvBlocks	Pushes the cache to the remote node in block list mode.
RegisterKvCache	Registers the local KV cache memory.
UnregisterKvCache	Deregisters the local KV cache memory.

**Table 2** HIXL APIs
API	Description
HIXL Constructor	Creates an HIXL object.
~Hixl()	Destructs an HIXL object.
Initialize	Initializes an HIXL instance. This API must be called before other APIs are called.
Finalize	Cleans up HIXL resources.
RegisterMem	Registers a memory address for use by TransferSync in subsequent calls. The local and remote memory addresses specified in TransferSync can be a subset of the registered addresses. Local memory addresses must be registered with the local HIXL instance, and remote memory addresses must be registered with the remote HIXL instance.
DeregisterMem	Deregisters memory.
Connect	Establishes a link with the remote HIXL instance.
Disconnect	Disconnects a link with the remote HIXL instance.
TransferSync	Transmits memory with the remote HIXL instance.
TransferAsync	Performs batch asynchronous transmission and submits transfer requests.
GetTransferStatus	Obtains the status of a transfer request.
SendNotify	Sends a notification message from the client to the server.
GetNotifies	Obtains all notification messages received by servers in the current HIXL instance and clears them.

**Table 3** ADXL APIs
API	Description
AdxlEngine Constructor	Creates an AdxlEngine object.
~AdxlEngine()	Destructs an AdxlEngine object.
Initialize	Initializes an AdxlEngine instance. This API must be called before other APIs are called.
Finalize	Cleans up AdxlEngine resources.
RegisterMem	Registers a memory address for use by TransferSync in subsequent calls. The local and remote memory addresses specified in TransferSync can be a subset of the registered addresses. Local memory addresses must be registered with the local AdxlEngine instance, and remote memory addresses must be registered with the remote AdxlEngine instance.
DeregisterMem	Deregisters memory.
Connect	Establishes a link with the remote AdxlEngine instance.
Disconnect	Disconnects a link with the remote AdxlEngine instance.
TransferSync	Transmits memory with the remote AdxlEngine instance.
SendNotify	Sends a notification message from the client to the server.
GetNotifies	Obtains all notification messages received by servers in the current AdxlEngine instance and clears them.
MallocMem	Allocates memory.
FreeMem	Deregisters memory.

Parent topic: API Reference