Triton Overview

Triton is a Python-based programming language that has gained popularity in recent years. You only need to focus on the tiling/block slicing mode and its computation logic. The Triton compiler can automatically handle tasks like memory allocation, data movement, computation scheduling, and pipeline parallelism during compilation, based on the underlying hardware features. This makes operator development easier with higher efficiency.

Triton-Ascend is a Triton compilation framework built for the Ascend platform, aiming to enable Triton code to run efficiently on Ascend hardware. We are actively improving Triton-Ascend by enhancing Python APIs, supporting more data types, making memory access more flexible, and optimizing the compiler's automatic optimization capabilities. These improvements aim to boost the overall functionality and performance of Triton-Ascend.

Integrating Triton operators into GE graphs allows you to reuse custom Triton operators developed during GPU training and incorporate them into the graph execution process with minimal costs. This gives you the best possible performance benefited from graph offloading.

The following figure shows the overall process.

The process consists of two phases: development and running.

• Development phase
  1. Configure the environment. For details, see Environment Setup.
  2. Compile the Triton operator kernel into the corresponding .npubin file. For details, see Generating an .npubin File.
  3. Develop the deliverable for integrating an operator into a TensorFlow graph by referring to Developing the Deliverable for Integrating an Operator into a TensorFlow Graph. Currently, only the TensorFlow framework is supported.
  4. Create the project for integrating an operator into the graph, develop the deliverable for integrating a Triton operator into a GE graph, and build and deploy the deliverable. For details, see Developing the Deliverable for Integrating an Operator into a GE Graph.
• Running phase: Call the TensorFlow custom operator in the normal way and use GE to parse a model into a graph. After graph build and run is complete, use graph offloading to significantly improve performance. For details, see Verifying the Result.