Batch Random Seed, Top-K logProbs Exponential Sampling

Each batch of the vocabulary probabilities is truncated by using a different top-k, and then exponential sampling of the top-p is performed on each batch of the truncated vocabulary probabilities by using a different random seed.

Flowchart

Figure 1 Batch random seed top-k logProbs exponential sampling process