sync_kv_timeout

Applicability

Product	Supported (√/x)
Atlas 350 Accelerator Card	x
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Note: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Sets the timeout interval for pull_cache, pull_blocks, push_blocks, and push_cache APIs, corresponding to the underlying llm.SyncKvCacheWaitTime configuration item.

Prototype

sync_kv_timeout(sync_kv_timeout)

Parameters

Parameter	Data Type	Description
sync_kv_timeout	int or str	KV synchronization timeout interval, in milliseconds. If the parameter is not configured, the interval 1000 ms is used.

Example

from llm_datadist import LLMConfig
llm_config = LLMConfig()
llm_config.sync_kv_timeout = 1000

Returns

In normal cases, no value is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Restrictions

None

Parent topic: LLMConfig