torch.nn

torch.nn

表1 torch.nn

序号

API名称

是否支持

1

Parameter

2

UninitializedParameter

Containers

表2 Containers

序号

API名称

是否支持

1

Module

2

ModuleDict

3

ModuleList

4

ParameterDict

5

ParameterList

6

register_module_backward_hook

7

register_module_forward_hook

8

register_module_forward_pre_hook

9

Sequential

Convolution Layers

表3 Convolution Layers

序号

API名称

是否支持

1

nn.Conv1d

2

nn.Conv2d

3

nn.Conv3d

4

nn.ConvTranspose1d

5

nn.ConvTranspose2d

6

nn.ConvTranspose3d

7

nn.Fold

8

nn.LazyConv1d

9

nn.LazyConv2d

10

nn.LazyConv3d

11

nn.LazyConvTranspose1d

12

nn.LazyConvTranspose2d

13

nn.LazyConvTranspose3d

14

nn.Unfold

Pooling layers

表4 Pooling layers

序号

API名称

是否支持

1

nn.AdaptiveAvgPool1d

2

nn.AdaptiveAvgPool2d

3

nn.AdaptiveAvgPool3d

是,仅支持D=1,H=1,W=1场景

4

nn.AdaptiveMaxPool1d

5

nn.AdaptiveMaxPool2d

6

nn.AdaptiveMaxPool3d

7

nn.AvgPool1d

8

nn.AvgPool2d

9

nn.AvgPool3d

10

nn.FractionalMaxPool2d

11

nn.LPPool1d

12

nn.LPPool2d

13

nn.MaxPool1d

14

nn.MaxPool2d

15

nn.MaxPool3d

16

nn.MaxUnpool1d

17

nn.MaxUnpool2d

18

nn.MaxUnpool3d

Padding Layers

表5 Padding Layers

序号

API名称

是否支持

1

nn.ConstantPad1d

2

nn.ConstantPad2d

3

nn.ConstantPad3d

4

nn.ReflectionPad1d

5

nn.ReflectionPad2d

6

nn.ReplicationPad1d

7

nn.ReplicationPad2d

8

nn.ReplicationPad3d

9

nn.ZeroPad2d

Non-linear Activations (weighted sum, nonlinearity)

表6 Non-linear Activations

序号

API名称

是否支持

1

nn.CELU

2

nn.ELU

3

nn.GELU

4

nn.Hardshrink

5

nn.Hardsigmoid

6

nn.Hardswish

7

nn.Hardtanh

8

nn.LeakyReLU

9

nn.LogSigmoid

10

nn.MultiheadAttention

11

nn.PReLU

12

nn.ReLU

13

nn.ReLU6

14

nn.RReLU

15

nn.SELU

16

nn.Sigmoid

17

nn.SiLU

18

nn.Softplus

19

nn.Softshrink

20

nn.Softsign

21

nn.Tanh

22

nn.Tanhshrink

23

nn.Threshold

Non-linear Activations (other)

表7 Normalization Layers

序号

API名称

是否支持

1

nn.AdaptiveLogSoftmaxWithLoss

2

nn.LogSoftmax

3

nn.Softmax

4

nn.Softmax2d

5

nn.Softmin

Normalization Layers

表8 Non-linear Activations

序号

API名称

是否支持

1

nn.BatchNorm1d

2

nn.BatchNorm2d

3

nn.BatchNorm3d

4

nn.GroupNorm

5

nn.InstanceNorm1d

6

nn.InstanceNorm2d

7

nn.InstanceNorm3d

8

nn.LayerNorm

9

nn.LocalResponseNorm

10

nn.SyncBatchNorm

Recurrent Layers

表9 Recurrent Layers

序号

API名称

是否支持

1

nn.GRU

2

nn.GRUCell

3

nn.LSTM

4

nn.LSTMCell

是,非16对齐场景暂不支持

5

nn.RNN

6

nn.RNNBase

7

nn.RNNCell

Transformer Layers

表10 Transformer Layers

序号

API名称

是否支持

1

nn.Transformer

2

nn.TransformerDecoder

3

nn.TransformerDecoderLayer

4

nn.TransformerEncoder

5

nn.TransformerEncoderLayer

Linear Layers

表11 Linear Layers

序号

API名称

是否支持

1

nn.Bilinear

2

nn.Identity

3

nn.LazyLinear

4

nn.Linear

Dropout Layers

表12 Dropout Layers

序号

API名称

是否支持

1

nn.AlphaDropout

2

nn.Dropout

3

nn.Dropout2d

4

nn.Dropout3d

Sparse Layers

表13 Sparse Layers

序号

API名称

是否支持

1

nn.Embedding

2

nn.EmbeddingBag

Distance Functions

表14 Distance Functions

序号

API名称

是否支持

1

nn.CosineSimilarity

2

nn.PairwiseDistance

Loss Functions

表15 Loss Functions

序号

API名称

是否支持

1

nn.BCELoss

2

nn.BCEWithLogitsLoss

3

nn.CosineEmbeddingLoss

4

nn.CrossEntropyLoss

5

nn.CTCLoss

6

nn.GaussianNLLLoss

7

nn.HingeEmbeddingLoss

8

nn.KLDivLoss

9

nn.L1Loss

10

nn.MarginRankingLoss

11

nn.MSELoss

12

nn.MultiLabelMarginLoss

13

nn.MultiLabelSoftMarginLoss

14

nn.MultiMarginLoss

15

nn.NLLLoss

16

nn.PoissonNLLLoss

17

nn.SmoothL1Loss

18

nn.SoftMarginLoss

19

nn.TripletMarginLoss

20

nn.TripletMarginWithDistanceLoss

Vision Layers

表16 Vision Layers

序号

API名称

是否支持

1

nn.PixelShuffle

2

nn.PixelUnshuffle

3

nn.Upsample

4

nn.UpsamplingBilinear2d

5

nn.UpsamplingNearest2d

Shuffle Layers

表17 Shuffle Layers

序号

API名称

是否支持

1

nn.ChannelShuffle

DataParallel Layers (multi-GPU, distributed)

表18 DataParallel Layers

序号

API名称

是否支持

1

nn.DataParallel

2

nn.parallel.DistributedDataParallel

Utilities

表19 Utilities

序号

API名称

是否支持

1

clip_grad_norm_

2

clip_grad_value_

3

nn.Flatten

4

nn.Unflatten

5

nn.utils.rnn.pack_padded_sequence

6

nn.utils.rnn.pack_sequence

7

nn.utils.rnn.PackedSequence

8

nn.utils.rnn.pad_packed_sequence

9

nn.utils.rnn.pad_sequence

10

parameters_to_vector

11

prune.BasePruningMethod

12

prune.custom_from_mask

13

prune.CustomFromMask

14

prune.global_unstructured

15

prune.Identity

16

prune.identity

17

prune.is_pruned

18

prune.l1_unstructured

19

prune.L1Unstructured

20

prune.ln_structured

21

prune.LnStructured

22

prune.PruningContainer

23

prune.random_structured

24

prune.random_unstructured

25

prune.RandomStructured

26

prune.RandomUnstructured

27

prune.remove

28

remove_spectral_norm

29

remove_weight_norm

30

spectral_norm

31

vector_to_parameters

32

weight_norm

Lazy Modules Initialization

表20 Lazy Modules Initialization

序号

API名称

是否支持

1

nn.modules.lazy.LazyModuleMixin