Configuring DCI Compilation Options to Reduce the Operator Tail Overhead

This performance optimization suggestion applies to the following models:

• Atlas 350 Accelerator Card

[Priority] High

[Description] Upon operator execution completion, the DCache must be invalidated to prevent subsequent operators from using its data and being affected. You can add --cce-no-dcache-flush=true to the compilation options to append the DataCacheInvalid (DCI) instruction to the end of the operators, thereby invalidating the DCache. If this option is not enabled, the DataCacheCleanAndInvalid (DCCI) instruction is added by default to invalidate the DCache.

Compared with inserting the DCCI instruction, inserting the DCI instruction eliminates the data synchronization process from the DCache to the GM (Clean), providing a certain performance advantage. Inserting the DCCI instruction provides an additional fault tolerance guarantee. If you modify the GM using the * __gm__ method or call the GlobalTensor.SetValue function without correctly calling the DataCacheCleanAndInvalid API to ensure cache consistency, the compilation framework automatically inserts the DCCI instruction to ensure normal operator precision.

Therefore, you can enable this compilation option to reduce the operator tail overhead in the following scenarios:

• When an operator modifies the GM using the * __gm__ method or calls the GlobalTensor.SetValue function, the DataCacheCleanAndInvalid API is correctly used to manually flush data from the DCache to the GM, ensuring cache consistency. Thus, consistency is ensured without relying on the compilation framework's automatic insertion of the DCCI instruction.
• An operator does not contain code that modifies the GM using the * __gm__ method or calls the GlobalTensor.SetValue function.