cache_desc

Applicability

Note: For the Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Obtains the KV cache description.

Prototype

1
2

@property
cache_desc() -> CacheDesc

Parameters

None

Example

1
2
3

...
kv_cache = kv_cache_manager.allocate_cache(cache_desc, cache_keys)
print(kv_cache.cache_desc.num_tensors)

Returns

In normal cases, the cache description of the KvCache type is returned.

Constraints

None