enable_remote_cache_accessible

Applicable Products

Note: For Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Specifies whether to enable the function of directly accessing the remote cache.

When this parameter is enabled, the remote cache metadata (such as index and memory address) is cached locally to accelerate the pull process. However, this also introduces relevant restrictions. For details, see Constraints.

Prototype

1

enable_remote_cache_accessible(self, enable_remote_cache_accessible: bool)

Parameters

If this parameter is disabled, Atlas A3 training products/Atlas A3 inference products support only the RDMA transmission protocol. You are advised to enable this parameter to support more types of transmission protocols.

For Ascend 950PR/Ascend 950DT, this parameter cannot be set to False.

Example

1
2
3

from llm_datadist import LLMConfig
llm_config = LLMConfig()
llm_config.enable_remote_cache_accessible = True

Returns

In normal cases, no value is returned.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Constraints

The index update is triggered only during the first pull. Subsequent cache operations on the remote side are not visible to the local side after that. Therefore, you need to ensure the validity of the cache. This parameter is more suitable for the PagedAttention scenario, where caches are allocated and registered only during the initialization phase of the user script and do not change frequently.