BlocksCacheKey

Applicability

Product	Supported (√/x)
Atlas 350 Accelerator Card	x
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product	x
Atlas training product	x

Note: For Atlas A2 training product/Atlas A2 inference product, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Constructs a BlocksCacheKey instance, which is usually used as the parameter type in allocate_blocks_cache, pull_blocks, and push_blocks of CacheManager.

Prototype

__init__(*args, **kwargs)

Parameters

Parameter	Data Type	Description
cluster_id	int	(Mandatory) ID of the remote cluster where the cache is located.
model_id	int	ID of the model associated with the cache. The default value is 0.

Example

from llm_datadist import BlocksCacheKey
blocks_cache_key = BlocksCacheKey(0, 0)

Returns

In normal cases, a BlocksCacheKey instance is returned.

If the input data type is incorrect, the TypeError or ValueError exception is reported.

Restrictions

None

Parent topic: API Reference