cache_manager

Applicable Products

Product	Supported (√/x)
Ascend 950PR/Ascend 950DT	√
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference products	x
Atlas training products	x

Note: For Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Obtains a CacheManager instance.

Prototype

1	cache_manager()

Parameters

None

Example

from llm_datadist import LLMDataDist, LLMRole
llm_datadist = LLMDataDist(LLMRole.DECODER, 0)
...
llm_datadist.init(engine_options)
cache_manager = llm_datadist.cache_manager

Returns

The CacheManager instance is returned.

Constraints

None

Parent topic: LLMDataDist