copy_cache

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference products	x
Atlas training products	x

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server, Atlas 300I A2 inference card, and A200I A2 Box heterogeneous components are supported.

Function Description

Copies a cache.

Prototype

copy_cache(dst: Cache, src: Cache, dst_batch_index: int = 0, src_batch_index: int = 0, offset: int = 0, size: int = -1, req_id: Optional[int] = None)

Parameters

Parameter	Data Type	Description
dst	Cache	Target cache.
src	Cache	Source cache.
dst_batch_index	int	Batch index of the destination cache. The default value is 0.
src_batch_index	int	Batch index of the source cache. The default value is 0.
offset	int	Offset of each tensor. The default value is 0.
size	int	If this parameter is set to an integer greater than 0, it indicates the size of the data to be copied. If this parameter is set to -1, it indicates a full copy. The default value is -1.
req_id	Optional[int]	Request ID associated with the current call. If this parameter is set, the request ID is recorded in the maintenance and debugging logs related to the local call. The default value is None.

Example

from llm_datadist import *
...
cache_manager.copy_cache(dst_cache, src_cache, 0, 1, 0, 128)

Returns

In normal cases, no value is returned.

If the input data type is incorrect, a TypeError or ValueError is thrown.

If the execution time exceeds the value of sync_kv_timeout, an LLMException is thrown.

Restrictions

CacheDesc of the source cache and destination cache must be matched.

Parent topic: CacheManager (to Be Deprecated)