Computable Batch Control

Description

Specifies batches for attention computation.

How to Enable

Set batchRunStatusEnable to true and pass batchRunStatus as the input tensor.

Tensor whose batchRunStatus is 0 or 1. The value 0 indicates that the batch is not involved in the computation, and the value 1 indicates that the batch is involved in the computation.

Constraints

Not supported by the Atlas inference products.