[object Object]

[object Object][object Object]undefined[object Object]

• Function: BatchMatMulReduceScatterAllToAll is a computing and communication fusion operator that implements parallelism of BatchMatMul computation and ReduceScatter and AllToAll collective communication.

• The calculation formula is as follows: BatchMatMul computation --> Transpose (required when yShardType is 0) --> ReduceScatter collective communication --> Add --> AllToAll collective communication. Below shows the computation logic, where [object Object] is the output.

$$temp1 = BatchMatMul(x, weight)$$ $$temp2 = ReduceScatter(temp1)$$ $$temp3 = Add(temp2, bias)$$ $$y = AllToAll(temp3)$$ [object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computing:

  • [object Object] defaults to a deterministic implementation.

  • Due to the requirements of collective communication and BatchMatMul computation, the input and output shapes must meet the following mathematical relationship (ep = epWorldSize and tp = tpWorldSize):

  • ReduceScatter by H axis (yShardType = 0):

    • x: (E/ep, ep*C, M/tp)
    • weight: (E/ep, M/tp, H)
    • biasOptional: (E/ep, 1, H/tp) for 3D and (E/ep, H/tp) for 2D when the pointer is not empty
    • y: (E, C, H/tp)

  • ReduceScatter scenario by the C axis, that is, the scenario where yShardType is 1:

    • x: (E/ep, eptpC/tp, M/tp)
    • weight: (E/ep, M/tp, H)
    • biasOptional: (E/ep, 1, H) for 3D and (E/ep, H) for 2D when the pointer is not empty
    • y: (E, C/tp, H)

• Data relationship description:

  • For example, if x.size(0) is equal to E/tp and y.size(0) is equal to E, it indicates that y.size(0) = ep*x.size(0), where y.size (0) is an integer multiple of ep. Other relationships are similar to this.
  • The value range of E is [2, 512], and E is an integer multiple of ep.
  • The value range of H is [1, 65535]. When yShardType is 0, H is an integer multiple of tp.
  • The value range of M/tp is [1, 65535].
  • The value range of E/ep is [1, 32].
  • ep and tp can only be 2, 4, 8, 16, or 32.
  • groupEp and groupTp cannot have the same name.
  • When C is greater than 0, its maximum value is the upper limit of the operator device memory. When yShardType is 1, C is an integer multiple of tp.
  • MC2 operators cannot be called concurrently, nor can different MC2 operators.
  • Cross-supernode operations are not supported.

[object Object]

The following example is for reference only. For details, see .

• [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  [object Object]