enable_remote_cache_accessible

Applicability

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server, Atlas 300I A2 inference card, and A200I A2 Box heterogeneous components are supported.

Function Description

Specifies whether to enable the function of directly accessing the remote cache.

When this option is enabled, the remote cache metadata (such as index and memory address) is cached locally to accelerate the pull process. However, this also introduces relevant restrictions. For details, see Restrictions.

Prototype

1

enable_remote_cache_accessible(self, enable_remote_cache_accessible: bool)

Parameters

Example

1
2
3

from llm_datadist import LLMConfig
llm_config = LLMConfig()
llm_config.enable_remote_cache_accessible = True

Returns

In normal cases, no value is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Restrictions

The index update is triggered only during the first pull. Subsequent cache operations on the remote side are not visible to the local side after that. Therefore, you are responsible for ensuring the validity of the cache. This option is more suitable for the PagedAttention scenario, where caches are allocated and registered only during the initialization phase of the user script and do not change frequently.