API List

LLM-DataDist is an LLM distributed cluster and data acceleration component. It provides clustered key-value (KV) data management capabilities and supports disaggregated deployment of full graphs and incremental graphs.
- LLM-DataDist APIs are stored in ${INSTALL_DIR}/include/llm_datadist/llm_datadist.h. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.
- The library file corresponding to LLM-DataDist APIs is libllm_datadist.so.
HIXL provides high-performance, zero-copy point-to-point data transmission capabilities for cluster scenarios and exposes its capabilities through simple APIs.
- HIXL APIs are stored in ${INSTALL_DIR}/include/hixl/hixl.h. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.
- The library file corresponding to HIXL APIs is libcann_hixl.so.
- HIXL_CS APIs are stored in ${INSTALL_DIR}/include/cs/hixl_cs.h. Replace ${INSTALL_DIR} with the CANN component directory. For example, if the installation is performed by the root user, the default file storage path is /usr/local/Ascend/cann.
- The library file corresponding to HIXL_CS APIs is libcann_hixl.so.

The following products are supported:

Atlas A2 training products/Atlas A2 inference products: For Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported. When the HCCS transmission protocol is used in the server, LLM-DataDist APIs support only D2D transmission.

Atlas A3 training products/Atlas A3 inference products: When the HCCS transmission protocol is used, LLM-DataDist APIs do not support the host memory as the remote cache.
Ascend 950PR/Ascend 950DT: The UB protocol is used within a SuperPoD, and the RoCE protocol is used between SuperPoDs.

The APIs are listed as follows.

**Table 1** LLM-DataDist APIs_V2
API	Description
LlmDataDist Constructor	Constructs an LLM-DataDist object.
~LlmDataDist()	Destructs an LLM-DataDist object.
Initialize	Initializes the LLM-DataDist instance.
Finalize	Releases the LLM-DataDist.
SetRole	Sets the role of the current LLM-DataDist.
LinkLlmClusters	Establishes a link.
UnlinkLlmClusters	Disconnects a link.
PullKvCache	Pulls the KV cache in contiguous memory mode.
PullKvBlocks	Pulls the KV cache in block list mode.
PushKvCache	Pushes the cache to the remote node.
PushKvBlocks	Pushes the cache to the remote node in block list mode.
RegisterKvCache	Registers the local KV cache memory.
UnregisterKvCache	Deregisters the local KV cache memory.

**Table 2** HIXL APIs
API	Description
HIXL Constructor	Creates an HIXL object.
~Hixl()	Destructs an HIXL object.
Initialize	Initializes an HIXL instance. This API must be called before other APIs are called.
Finalize	Cleans up HIXL resources.
RegisterMem	Registers a memory address for use by TransferSync in subsequent calls. The local and remote memory addresses specified in TransferSync can be a subset of the registered addresses. Local memory addresses must be registered with the local HIXL instance, and remote memory addresses must be registered with the remote HIXL instance.
DeregisterMem	Deregisters memory.
Connect	Establishes a link with the remote HIXL instance.
Disconnect	Disconnects a link with the remote HIXL instance.
TransferSync	Transmits memory with the remote HIXL instance.
TransferAsync	Performs batch asynchronous transmission and submits transfer requests.
GetTransferStatus	Obtains the status of a transfer request.
SendNotify	Sends a Notify message from the client to the server.
GetNotifies	Obtains all Notify messages received by servers in the current HIXL instance and clears them.

**Table 3** HIXL CS APIs
API	Description
HixlCSServerCreate	Creates and initializes a server instance.
HixlCSServerRegMem	Registers the shared memory by the server for the client to access.
HixlCSServerListen	Starts server listening to accept connections from the client.
HixlCSServerUnregMem	Deregisters the memory that has been registered.
HixlCSServerDestroy	Destroys a server instance and releases related resources.
HixlCSClientCreate	Creates a client instance and initializes local resources.
HixlCSClientConnect	Initiates synchronous connection establishment with the server (blocking call until success or timeout).
HixlCSClientGetRemoteMem	Obtains the information about the memory registered by the server.
HixlCSClientRegMem	Registers the local memory for the client.
HixlCSClientUnregMem	Deregisters the memory registered by the client.
HixlCSClientBatchPutAsync	Writes multiple groups of data to the server asynchronously in batches.
HixlCSClientBatchGetAsync	Reads multiple groups of data from the server to the local host asynchronously in batches.
HixlCSClientQueryCompleteStatus	Queries the completion status of an asynchronous batch task.
HixlCSClientDestroy	Destroys a client instance and releases resources.

**Table 4** ADXL APIs
API	Description
AdxlEngine Constructor	Creates an AdxlEngine object.
~AdxlEngine()	Destructs an AdxlEngine object.
Initialize	Initializes an AdxlEngine instance. This API must be called before other APIs are called.
Finalize	Cleans up AdxlEngine resources.
RegisterMem	Registers a memory address for use by TransferSync in subsequent calls. The local and remote memory addresses specified in TransferSync can be a subset of the registered addresses. Local memory addresses must be registered with the local AdxlEngine instance, and remote memory addresses must be registered with the remote AdxlEngine instance.
DeregisterMem	Deregisters memory.
Connect	Establishes a link with the remote AdxlEngine instance.
Disconnect	Disconnects a link with the remote AdxlEngine instance.
TransferSync	Transmits memory with the remote AdxlEngine instance.
SendNotify	Sends a Notify message from the client to the server.
GetNotifies	Obtains all Notify messages received by servers in the current AdxlEngine instance and clears them.
MallocMem	Allocates memory.
FreeMem	Deregisters memory.

Parent topic: API Reference