deallocate_cache

Applicability

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server, Atlas 300I A2 inference card, and A200I A2 Box heterogeneous components are supported.

Function Description

Deallocates a cache.

If the cache is associated with a cache key during allocation, the actual deallocation is delayed until all cache keys are pulled or remove_cache_key is executed.

Prototype

1

deallocate_cache(cache: Cache)

Parameters

Example

1
2
3

from llm_datadist import *
...
cache_manager.deallocate_cache(cache)

Returns

In normal cases, no value is returned.

If the input data type is incorrect, a TypeError or ValueError is thrown.

If the execution time exceeds the value of sync_kv_timeout, an LLMException is thrown.

Restrictions

If Cache does not exist or has been released, this operation is a no-operation.