remove_cache_key

Applicability

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server, Atlas 300I A2 inference card, and A200I A2 Box heterogeneous components are supported.

Function Description

Removes a cache key.

After a cache key is removed, the corresponding cache can no longer be pulled by pull_cache.

Prototype

1

remove_cache_key(cache_key: CacheKey)

Parameters

Example

1
2
3
4
5

from llm_datadist import *
...
cache_keys = [CacheKey(1, req_id=1), CacheKey(1, req_id=2)]
cache_manager.remove_cache_key(cache_keys[0])
cache_manager.remove_cache_key(cache_keys[1])

Returns

In normal cases, no value is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

If the execution time exceeds the value of sync_kv_timeout, an LLMException is thrown.

Restrictions

If CacheKey does not exist or has been removed, this operation is a no-operation.