Description

The leading dimension function is supported. The key and value are determined by the passed tensors kStrides, vStrides, kOffset, and vOffset. kStrides and vStrides indicate the stride from the current element to the next element in the memory. kOffset indicates the offset of the position of the first element in the key relative to the start address of the input. vOffset indicates the offset of the position of the first element in the value relative to the start address of the input.

Traverse each key and value, and fills the key and value(num_heads, head_size) in the specified position of key_cache/value_cache with slotmapping.

Figure 1 ReshapeAndCacheWithStrideOperation function diagram