SetL2CacheHint

Applicability

Product	Supported
Atlas 350 Accelerator Card	√
Atlas A3 training product/Atlas A3 inference product	√
Atlas A2 training product/Atlas A2 inference product	√
Atlas 200I/500 A2 inference product	x
Atlas inference product AI Core	x
Atlas inference product Vector Core	x
Atlas training product	x

Function Usage

Sets whether to enable the L2 cache for the GlobalTensor. By default, the L2 cache is enabled.

Prototype

template<CacheRwMode rwMode = CacheRwMode::RW>
__aicore__ inline void SetL2CacheHint(CacheMode mode);

Parameters

Table 1 Template parameters

Parameter

Description

rwMode

Read/write mode of the L2 cache.

enum CacheRwMode {
READ = 1,
WRITE = 2,
RW = 3
};

Reserved parameter. This parameter is reserved for future functions. You can use the default value.

Table 2 Parameters

Parameter

Input/Output

Description

mode

Input

L2 cache mode specified by the user.

enum class CacheMode : uint8_t {
CACHE_MODE_DISABLE = 0, // Disable the L2 cache.
CACHE_MODE_NORMAL = 1,  // Enable the L2 cache.
};

If enabling the L2 cache for a GlobalTensor during operator writing results in lower performance than disabling the L2 cache, you can manually disable the L2 cache for the GlobalTensor. For example, if an operator reads a GlobalTensor only once, loading the data into the L2 cache does not benefit the operator. Instead, frequent data movement to the L2 cache may cause performance loss. In this case, you can consider disabling the L2 cache for the GlobalTensor.

If this API is not called, the default value CacheMode::CACHE_MODE_NORMAL is used, meaning that the L2 cache is enabled for the GlobalTensor.

Returns

None

Restrictions

When the mssanitizer tool is used, the L2 cache is enabled by default. In this case, the L2 cache mode cannot be set to CACHE_MODE_DISABLE by using this API.

Examples

uint64_t dataSize = 256; // Set the size of input_global to 256.

AscendC::GlobalTensor<int32_t> inputGlobal; // The type is int32_t.
inputGlobal.SetGlobalBuffer(reinterpret_cast<__gm__ int32_t *>(src_gm), dataSize); // Set the start address of the source operand in the global memory to src_gm and the size of the external storage occupied by the source operand to 256 int32_t data elements.
inputGlobal.SetL2CacheHint(AscendC::CacheMode::CACHE_MODE_DISABLE); // Specify that the GlobalTensor will not be written to the L2 cache.

AscendC::LocalTensor<int32_t> inputLocal = inQueueX.AllocTensor<int32_t>();    
AscendC::DataCopy(inputLocal, inputGlobal, dataSize); // Copy inputGlobal from the global memory to inputLocal of the local memory.

Parent topic: GlobalTensor