[object Object][object Object][object Object]undefined[object Object]

• Operator function: Fetches discontinuous tokens from [object Object]/[object Object] and assembles them into a continuous [object Object]/[object Object] sequence based on the [object Object] value in [object Object] and [object Object] of [object Object]/[object Object] from [object Object].

• Computation logic:

  • The first dimension of [object Object]/[object Object] depends on the value of [object Object].
  • If [object Object] is [object Object], the last value in [object Object] is the size of the first dimension of [object Object]/[object Object]: [object Object] = [object Object].
  • If [object Object] is [object Object], the sum of all values in [object Object] is the size of the first dimension of [object Object]/[object Object]: [object Object] = [object Object].

  Restrictions on [object Object] and [object Object]:

  • The size of each token must be less than 148k. For example, for the FP16/BF16 type, [object Object] × [object Object] ([object Object]/[object Object]) is 128 × 576.

• Example:

  [object Object]

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

• [object Object]
• [object Object]

[object Object]

• Parameters:

  • [object Object] ([object Object], an input parameter): Device-side [object Object], indicating the [object Object] vector cache stored at the current layer. When [object Object] is [object Object], the shape is [object Object]. When [object Object] is [object Object], the shape is [object Object]. ([object Object] is [object Object] in the b8 scenario, [object Object] in the b16 scenario, and [object Object] in the b32 scenario. In the b8 scenario, the bit width of each data element is 8 bits, for example, INT8. In the b16 scenario, the bit width of each data element is 16 bits, for example, INT16. In the b32 scenario, the bit width of each data element is 32 bits, for example, INT32.) and empty tensors are not supported.

    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas 200I/500 A2 inference products[object Object], [object Object]Atlas inference products[object Object], and [object Object]Atlas training products[object Object]: The data type can be INT8, FLOAT16, or BFLOAT16. The can be ND.

  • [object Object] ([object Object], an input parameter): Device-side [object Object], indicating the [object Object] vector cache stored at the current layer. When [object Object] is [object Object], the shape is [object Object]. When [object Object] is [object Object], the shape is [object Object]. ([object Object] is [object Object] in the b8 scenario, [object Object] in the b16 scenario, and [object Object] in the b32 scenario.) and empty tensors are not supported.

    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas 200I/500 A2 inference products[object Object], [object Object]Atlas inference products[object Object], and [object Object]Atlas training products[object Object]: The data type can be INT8, FLOAT16, or BFLOAT16. The can be ND.

  • [object Object] ([object Object], an input parameter): Device-side [object Object], indicating the physical block index corresponding to each sequence. The shape is [object Object], and the element value range is [object Object]. and empty tensors are not supported.

    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas 200I/500 A2 inference products[object Object], [object Object]Atlas inference products[object Object], and [object Object]Atlas training products[object Object]: The data type can be int32_t. The can be ND.

  • [object Object] ([object Object], an input parameter): Device-side [object Object], indicating the sequence length corresponding to each batch. The can be ND, and the shape is [object Object] or [object Object]. When [object Object] is [object Object], the shape is [object Object]. When [object Object] is [object Object], the shape is [object Object]. The element value range is [object Object]. and empty tensors are not supported.

    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas 200I/500 A2 inference products[object Object], [object Object]Atlas inference products[object Object], and [object Object]Atlas training products[object Object]: The data type can be INT32. The can be ND.

  • [object Object] ([object Object], an input/output parameter): Device-side [object Object], indicating the [object Object] vector. The can be ND. When [object Object] is [object Object], the shape is [object Object]. When [object Object] is [object Object], the shape is [object Object]. and empty tensors are not supported.

    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas 200I/500 A2 inference products[object Object], [object Object]Atlas inference products[object Object], and [object Object]Atlas training products[object Object]: The data type can be INT8, FLOAT16, or BFLOAT16. The can be ND.

  • [object Object] ([object Object], an input/output parameter): Device-side [object Object], indicating the [object Object] vector. The can be ND. When [object Object] is [object Object], the shape is [object Object]. When [object Object] is [object Object], the shape is [object Object]. and empty tensors are not supported.

    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas 200I/500 A2 inference products[object Object], [object Object]Atlas inference products[object Object], and [object Object]Atlas training products[object Object]: The data type can be INT8, FLOAT16, or BFLOAT16. The can be ND.

  • [object Object] ([object Object], an optional input parameter): Device-side [object Object]. The can be ND, and the shape is [object Object]. If this parameter is passed, there is an initial offset when obtaining [object Object] from [object Object] (the offset is [object Object]/[object Object], where [object Object] indicates a batch). If this parameter is not passed, no offset is required. and empty tensors are not supported.

    • [object Object]Atlas A3 training products/Atlas A3 inference products[object Object], [object Object]Atlas A2 training products/Atlas A2 inference products[object Object], [object Object]Atlas 200I/500 A2 inference products[object Object], [object Object]Atlas inference products[object Object], and [object Object]Atlas training products[object Object]: The data type can be INT32. The can be ND.

  • [object Object] ([object Object], an input parameter): The options include [object Object] and [object Object], indicating the ND and NZ data formats, respectively. This parameter indicates the shape input mode of [object Object]/[object Object]/[object Object]/[object Object].

  • [object Object] ([object Object], an input parameter): Whether [object Object] is a cumulative sum. If [object Object] is [object Object], [object Object] is a non-cumulative sum. For example, [object Object] is [object Object]. If [object Object] is [object Object], [object Object] is a cumulative sum. For example, [object Object] is [object Object], and the 0th element must be [object Object]. The cumulative sum of [object Object] is equal to the non-cumulative [object Object].

  • [object Object] ([object Object], an output parameter): Size of the workspace to be allocated on the device.

  • [object Object] ([object Object], an output parameter): Operator executor, containing the operator computation process.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Parameters:

  • [object Object] ([object Object], an input parameter): Address of the workspace to be allocated on the device.

  • [object Object] ([object Object], an input parameter): Size of the workspace to be allocated on the device, which is obtained by the first-phase API [object Object].

  • [object Object] ([object Object], an input parameter): Operator executor, containing the operator computation process.

  • [object Object] ([object Object], an input parameter): Stream for executing the task.

• Returns:

  [object Object] status code. For details, see .

[object Object]

• Deterministic computation:
  • [object Object] defaults to a deterministic implementation.

[object Object]

The following example is for reference only. For details, see .