torch.distributed

Initialization

表1 Initialization
序号	API名称	是否支持
1	torch.distributed.init_process_group	是
2	torch.distributed.is_available	是
3	torch.distributed.is_initialized	是
4	torch.distributed.is_mpi_available	是
5	torch.distributed.is_nccl_available	是
6	torch.distributed.is_torchelastic_launched	否

Post-Initialization

表2 Post-Initialization
序号	API名称	是否支持
1	torch.distributed.Backend	是
2	torch.distributed.get_backend	是
3	torch.distributed.get_rank	是
4	torch.distributed.get_world_size	是

Distributed Key-Value Store

表3 Distributed Key-Value Store
序号	API名称	是否支持
1	torch.distributed.FileStore	否
2	torch.distributed.HashStore	否
3	torch.distributed.PrefixStore	否
4	torch.distributed.Store	否
5	torch.distributed.Store.add	否
6	torch.distributed.Store.compare_set	否
7	torch.distributed.Store.delete_key	否
8	torch.distributed.Store.get	否
9	torch.distributed.Store.num_keys	否
10	torch.distributed.Store.set	否
11	torch.distributed.Store.set_timeout	否
12	torch.distributed.Store.wait	否
13	torch.distributed.TCPStore	否

Groups

表4 Groups
序号	API名称	是否支持
1	torch.distributed.new_group	是

Point-to-point communication

表5 Point-to-point communication
序号	API名称	是否支持
1	torch.distributed.irecv	否
2	torch.distributed.isend	否
3	torch.distributed.recv	否
4	torch.distributed.send	否

Synchronous and asynchronous collective operations

表6 Synchronous and asynchronous collective operations
序号	API名称	是否支持
1	get_future	否
2	is_completed	是
3	wait	是

Collective functions

表7 Collective functions
序号	API名称	是否支持
1	torch.distributed.all_gather	是
2	torch.distributed.all_gather_object	是
3	torch.distributed.all_reduce	是
4	torch.distributed.all_to_all	否
5	torch.distributed.barrier	是
6	torch.distributed.broadcast	是
7	torch.distributed.broadcast_object_list	是
8	torch.distributed.gather	否
9	torch.distributed.gather_object	否
10	torch.distributed.monitored_barrier	否
11	torch.distributed.reduce	否
12	torch.distributed.reduce_op	是
13	torch.distributed.reduce_scatter	否
14	torch.distributed.ReduceOp	是
15	torch.distributed.scatter	否
16	torch.distributed.scatter_object_list	否

Profiling Collective Communication

表8 Profiling Collective Communication
序号	API名称	是否支持
1	torch.autograd.profiler	否
2	torch.profiler	否

Multi-GPU collective functions

表9 Multi-GPU collective functions
序号	API名称	是否支持
1	torch.distributed.all_gather_multigpu	否
2	torch.distributed.all_reduce_multigpu	否
3	torch.distributed.broadcast_multigpu	否
4	torch.distributed.reduce_multigpu	否
5	torch.distributed.reduce_scatter_multigpu	否

Launch utility

**表10** Launch utility
序号	API名称	是否支持
1	torch.distributed.launch	是

Spawn utility

**表11** Spawn utility
序号	API名称	是否支持
1	torch.multiprocessing.spawn	是

父主题： PyTorch1.11.0