copy_blocks

Applicability

Product	Supported (Yes/No)
Atlas 350 Accelerator Card	No
Atlas A3 training product/Atlas A3 inference product	Yes
Atlas A2 training product/Atlas A2 inference product	Yes
Atlas 200I/500 A2 inference product	No
Atlas inference product	No
Atlas training product	No

Note: For the Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Copies blocks in the PagedAttention scenario.

Prototype

copy_blocks(cache: KvCache, copy_block_info: Dict[int, List[int]])

Parameters

Parameter	Data Type	Description
cache	KvCache	Target cache.
copy_block_info	Dict[int, List[int]]	A dictionary mapping original block indices to lists of target block indices.

Example

from llm_datadist import *
...
kv_cache_manager.copy_blocks(kv_cache, {1: [2,3]})

Returns

In normal cases, no value is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

If the execution time exceeds the value of sync_kv_timeout, an LLMException is thrown.

Constraints

This API does not support concurrent calls. Concurrent calls will be queued.

Parent topic: KvCacheManager