LLMStatusCode

LLM_SUCCESS

Success

None

None

LLM_FAILED

Common failure

No

Restart the host or container.

Preserve the environment, collect host/device logs, and back them up.

LLM_PARAM_INVALID

Incorrect parameter

Yes

Locate the fault based on logs.

LLM_KV_CACHE_NOT_EXIST

KV not found

Yes

• Check whether the request in the full error log is complete.
• Check whether repeated pulling occurs.
• Check whether the parameters for marking the target cache are correct.

LLM_REPEAT_REQUEST

Repeated request

Yes

Check whether repeated calling exists.

LLM_NOT_YET_LINK

No link established

Yes

Verify the link status between the Decode and Prompt sides at the upper layer.

LLM_ALREADY_LINK

Link already established

Yes

Verify the link status between the Decode and Prompt sides at the upper layer.

LLM_LINK_FAILED

Link establishment failed

Yes

If this error code is returned in the second return value of link_clusters, check the network connection between the corresponding clusters.

LLM_UNLINK_FAILED

Link disconnection failed

Yes

If this error code is returned in the second return value of unlink_clusters, check the network connection between the corresponding clusters.

LLM_NOTIFY_PROMPT_UNLINK_FAILED

Failed to notify the Prompt side of link disconnection

Yes

1. Check the network connection between the Decode and Prompt.
2. Call unlink_clusters on the Prompt side to clean up residual resources.

LLM_CLUSTER_NUM_EXCEED_LIMIT

Excessive clusters

Yes

Check the input parameters of link_clusters and unlink_clusters and ensure that the number of clusters must not exceed 16.

LLM_PROCESSING_LINK

Link establishment in progress

Yes

A link establishment or disconnection operation is in progress. Try again later.

LLM_PREFIX_ALREADY_EXIST

Prefix already loaded

Yes

Check whether the public prefix with the same prefix ID has been loaded. If yes, release the resources first.

LLM_PREFIX_NOT_EXIST

Prefix not found

Yes

Check whether the prefix ID in the request has been loaded.

LLM_EXIST_LINK

Unreleased links when switch_role is called

Yes

Check whether unlink_clusters is called to disconnect all links before the role of the current LLM-DataDist instance is switched.

LLM_FEATURE_NOT_ENABLED

Feature not enabled

Yes

Check whether required parameters are passed in during LLM-DataDist initialization.

If this exception is thrown when the role of the current LLM-DataDist instance is switched, check whether enable_switch_role is set to True in LLMConfig during initialization.

LLM_TIMEOUT

Processing timeout

Yes

• If this error is reported by transmission-related APIs such as pull_cache, pull_blocks, and transfer_cache_async, the link is unrecoverable and must be re-established.
• If this error is reported by other APIs, increase the timeout interval and try again.

LLM_LINK_BUSY

Link busy

Yes

Check whether the APIs called at the same time conflict with each other. For example, this error code is reported when the following APIs are called at the same time:

• unlink and

  pull_cache

• pull_cache and transfer_cache_async over the same link

LLM_OUT_OF_MEMORY

Insufficient memory

Yes

Check whether the memory pool is sufficient for the requested KV size.

Check whether the allocated memory is released.

LLM_DEVICE_MEM_ERROR

Faulty virtual address of a memory UCE

Yes

Obtain the faulty virtual address of the memory UCE and rectify it by referring to the description of the torch_npu.npu.restart_device API in Ascend Extension for PyTorch Custom API Reference. If it is a KV cache memory, additionally call the remap_registered_memory API of the cache manager to restore the KV cache memory registered with the NIC.

Note: This error code is reserved and not supported currently.

LLM_SUSPECT_REMOTE_ERROR

Suspected UCE memory fault

No

The upper-layer framework must perform a comprehensive diagnosis in combination with other faults to determine whether the issue is a UCE memory fault or another type of fault.

LLM_UNKNOWN_ERROR

Unknown error

No

Preserve the environment, collect host/device logs, and back them up.