LLMStatusCode

LLM_SUCCESS

Success.

None

None

LLM_FAILED

Common failure.

No

Restart the host or container.

Keep the scene, obtain the host/device logs, and back up the logs.

LLM_WAIT_PROCESS_TIMEOUT

Process timed out.

Yes

• If this error is reported by transmission-related APIs such as pull_cache, pull_blocks, and transfer_cache_async, the link is unrecoverable and must be re-established.
• If this error is reported by other APIs, increase the timeout interval and try again.

LLM_PARAM_INVALID

Incorrect parameter.

Yes

Locate the fault based on logs.

LLM_KV_CACHE_NOT_EXIST

The KV does not exist.

Yes

• Check whether the request in the full error log is complete.
• Check whether repeated pulling occurs.
• Check whether the parameters for marking the target cache are correct.

LLM_REPEAT_REQUEST

Repeated request.

Yes

Check whether repeated calling exists.

LLM_NOT_YET_LINK

No link is established.

Yes

Verify the link status between the Decode and Prompt sides at the upper layer.

LLM_ALREADY_LINK

A link has been established.

Yes

Verify the link status between the Decode and Prompt sides at the upper layer.

LLM_LINK_FAILED

Link establishment failure.

Yes

If this error code is returned in the second return value of link_clusters, check the network connectivity between the corresponding clusters.

LLM_UNLINK_FAILED

Link disconnection failure.

Yes

If this error code is returned in the second return value of unlink_clusters, check the network connectivity between the corresponding clusters.

LLM_NOTIFY_PROMPT_UNLINK_FAILED

Failed to notify the Prompt side of link disconnection

Yes

1. Check the network connectivity between the Decode and Prompt sides.
2. Call unlink_clusters on the Prompt side to clean up residual resources.

LLM_CLUSTER_NUM_EXCEED_LIMIT

The number of clusters exceeds the upper limit.

Yes

Check the input parameters of link_clusters and unlink_clusters and ensure that the number of clusters must not exceed 16.

LLM_PROCESSING_LINK

Link establishment is under process.

Yes

A link establishment or disconnection operation is in progress. Try again later.

LLM_PREFIX_ALREADY_EXIST

The prefix already exists.

Yes

Check whether the public prefix with the same prefix ID has been loaded. If yes, release the resources first.

LLM_PREFIX_NOT_EXIST

The prefix is not found.

Yes

Check whether the prefix ID in the request has been loaded.

LLM_DEVICE_OUT_OF_MEMORY

Insufficient device memory

Yes

Check whether the allocated memory has been properly released.

LLM_EXIST_LINK

Unreleased links exist when switch_role is called.

Yes

Check whether unlink_clusters is called to disconnect all links before the role of the current LLM-DataDist instance is switched.

LLM_FEATURE_NOT_ENABLED

Feature (reserved)

Yes

Check whether required options are passed in during LLM-DataDist initialization.

If this exception is thrown when the role of the current LLM-DataDist instance is switched, check whether enable_switch_role is set to True in LLMConfig during initialization.

LLM_LINK_BUSY

Busy link.

Yes

Check whether the APIs called at the same time conflict with each other. For example, this error code is reported when the following APIs are called at the same time:

• pull_cache and transfer_cache_async of KvCacheManager over the same link
• check_link_status and pull_cache of KvCacheManager

LLM_OUT_OF_MEMORY

Insufficient memory.

Yes

This error code is displayed only in CacheManager mode.

Check whether the memory pool is sufficient for the requested KV size.

Check whether the allocated memory has been properly released.

LLM_DEVICE_MEM_ERROR

Faulty virtual address of an uncorrectable memory error (UCE)

Yes

Obtain the faulty virtual address of the memory UCE and rectify it by referring to the description of the torch_npu.npu.restart_device API in TorchNPU Custom APIs.

Note: This error code is reserved and not supported currently.

LLM_SUSPECT_REMOTE_ERROR

Suspected UCE memory fault

No

The upper-layer framework must perform a comprehensive diagnosis in combination with other faults to determine whether the issue is a UCE memory fault or another type of fault.

LLM_UNKNOWN_ERROR

Unknown error.

No

Keep the scene, obtain the host/device logs, and back up the logs.