[object Object]

[object Object][object Object]undefined[object Object]

• The API is used to fuse and parallelize AllToAll, AllGather collective communication, and BatchMatMul computation.

• Formula:

  The computing logic is as follows, where y1, y2, and y3 are the outputs.

  $$x1 = AllToAll(x)$$ $$y2 = AllGather(x1)$$ $$y3 = BatchMatMul(y2, weight, bias)$$ $$y1 = Activation(y3)$$

[object Object]

Each operator has calls. First, [object Object] is called to obtain the workspace size required for computation and the executor that contains the operator computation process. Then, [object Object] is called to perform computation.

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

  The first-phase API implements input parameter verification. The following errors may be thrown.

  [object Object]

[object Object]

• Parameters

  [object Object]

• Returns

  [object Object]: status code. For details, see .

[object Object]

• Deterministic computing:

  • [object Object] defaults to a deterministic implementation.

• Due to the requirements of collective communication and BatchMatMul computation, the input and output shapes must meet the following mathematical relationship (ep = epWorldSize and tp = tpWorldSize):

  • AllGather by the H axis (xShardType = 0):

    • x: (E, C, H/tp)
    • weight: (E/ep, H, M/tp)
    • biasOptional: (E/ep, 1, M/tp) for 3D and (E/ep, M/tp) for 2D when the pointer is not null.
    • y1Out: (E/ep, ep*C, M/tp)
    • y2OutOptional: (E/ep, ep*C, H)
    • y3OutOptional: (E/ep, ep*C, M/tp)

  • AllGather by the C axis (xShardType = 1):

    • x: (E, C/tp, H)
    • weight: (E/ep, H, M/tp)
    • biasOptional: (E/ep, 1, M/tp) for 3D and (E/ep, M/tp) for 2D when the pointer is not null.
    • y1Out: (E/ep, ep*tp*C/tp, M/tp)
    • y2OutOptional: (E/ep, ep*tp*C/tp, H)
    • y3OutOptional: (E/ep, ep*tp*C/tp, M/tp)

• Data relationship description:

  • For example, if x.size(0) is equal to E and weight.size(0) is equal to E/ep, it indicates that x.size(0) = ep*weight.size(0), where x.size(0) is an integer multiple of ep. Other relationships are similar to this.
  • The value range of E is [2, 512], and E is an integer multiple of ep.
  • The value range of H is [1, 65535]. When xShardType is set to 0, H is an integer multiple of tp.
  • The value range of M/tp is [1, 65535].
  • The value range of E/ep is [1, 32].
  • ep and tp can only be 2, 4, 8, 16, or 32.
  • groupEp and groupTp cannot have the same name.
  • C must be greater than 0 and less than or equal to the upper limit of the operator device memory. When xShardType is set to 1, C is an integer multiple of tp.
  • MC2 operators cannot be called concurrently, nor can different MC2 operators.
  • Cross-supernode operations are not supported.

[object Object]

The following example is for reference only. For details, see .

• [object Object]Atlas A3 training products/Atlas A3 inference products[object Object]:

  [object Object]