LayerSynchronizer

An abstract class that users need to inherit from and implement the related APIs. Currently, this class contains only the synchronize_layer API.

Applicable Products

Product	Supported (Yes/No)
Ascend 950PR/Ascend 950DT	No
Atlas A3 training products/Atlas A3 inference products	Yes
Atlas A2 training products/Atlas A2 inference products	Yes
Atlas 200I/500 A2 inference products	No
Atlas inference products	No
Atlas training products	No

Note: For the Atlas A2 training products/Atlas A2 inference products, only the Atlas 800I A2 inference server and A200I A2 Box heterogeneous subrack are supported.

Function Description

Waits until the execution of the specified layer of the model is complete. You need to inherit LayerSynchronizer and implement this API.

This API is called during the execution of transfer_cache_async in KvCacheManager. When the API returns a success status, the cache transmission for the current layer begins.

Prototype

synchronize_layer(layer_index: int, timeout_in_millis: Optional[int]) -> bool

Parameters

Parameter	Data Type	Value Description
layer_index	int	Index of a layer.
timeout_in_millis	Optional[int]	Timeout interval, which is not supported currently.

Example

This API is not directly called by users. Instead, it is called by KvCacheManager as a callback function.

Returns

In normal cases, a message is returned, indicating that whether the synchronization is successful.

If a parameter is incorrect, a TypeError or ValueError may be thrown.

Constraints

None

Parent topic: API Reference