CacheKey

Applicability

Product	Supported
Atlas A3 training products/Atlas A3 inference products	√
Atlas A2 training products/Atlas A2 inference products	√
Atlas 200I/500 A2 inference products	x
Atlas inference products	x
Atlas training products	x

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server, Atlas 300I A2 inference card, and A200I A2 Box heterogeneous components are supported.

Function Description

Constructs a CacheKey object, which is usually used as the parameter type in allocate_cache and pull_cache of CacheManager.

Prototype

__init__(*args, **kwargs)

Parameters

Parameter	Data Type	Description
cluster_id	int	(Mandatory) ID of the remote cluster where the cache is located.
req_id	int	(Mandatory) Request ID associated with the cache.
model_id	int	ID of the model associated with the cache. The default value is 0.
prefix_id	int	Public prefix ID associated with the cache. The default value is 2⁶⁴ – 1.

Example

from llm_datadist import CacheKey
cache_key = CacheKey(0, 1, 0)

Returns

In normal cases, a CacheKey instance is returned.

If the input data type is incorrect, a TypeError or ValueError is thrown.

Restrictions

None

Parent topic: HIXL API Reference (Python)