PadWithHiddenStateOperation
Description
PadWithHiddenState is a pad with a hidden layer. This operator is used to fill a variable-length sequence with a fixed length to facilitate batch processing and model training.
Definition
struct PadWithHiddenStateParam {
std::vector<int32_t> qSeqLen;
int32_t maxSeqLen = 4096;
uint8_t rsv[12] = {0};
};
Parameters
Member |
Type |
Default Value |
Description |
|---|---|---|---|
qSeqLen |
std::vector<int32_t> |
- |
Actual input length of each batch. The number of elements is batchSize. The value of batchSize cannot exceed 32. |
maxSeqLen |
int32_t |
- |
Maximum input length in qSeqLen. The value cannot exceed 4096. |
rsv[12] |
uint8_t |
{0} |
Reserved |
Input
Parameter |
Dimension |
Data Type |
Format |
Description |
|---|---|---|---|---|
x |
[nTokens, hiddenSize] |
float16 |
ND |
Tensor without redundancy before pad. The shape is the same as that of the unpad output. |
- Formula:
This formula is used to calculate the total number of tokens in all sequences in a batch.
- Example:
batchSize = 3
qSeqlen = [5, 10, 7]
nTokens = 5 +10 + 7 = 22
Output
Parameter |
Dimension |
Data Type |
Format |
Description |
|---|---|---|---|---|
output |
[batchSize, maxSeqLen, hiddenSize] |
float16 |
ND |
Redundant tensor after pad. The shape is the same as that of the unpad input. |