Generative AI Leader · Free Practice Question Medium
Question 23
A developer is using a generative AI model for creative writing. They want to ensure that while the model produces diverse outputs, it avoids generating extremely unlikely or nonsensical word choices. They prefer to limit the model's selection to a smaller set of more probable words that collectively account for a certain probability mass.
Which sampling parameter allows for this kind of control by considering the cumulative probability of tokens?
-
A
Top-k sampling
-
B
Top-p (nucleus) sampling
-
C
Output length
-
D
Temperature
Reveal correct answer
Correct answer: B
Explanation
Top-p (nucleus) sampling selects from the smallest set of tokens whose cumulative probability exceeds a given threshold 'p', providing a balance between diversity and coherence.
INCORRECT: Temperature
Temperature adjusts the randomness of the entire probability distribution. While related, top-p offers a more direct way to control the size of the token consideration set based on cumulative probability.INCORRECT: Top-k sampling
Top-k sampling considers only the 'k' most probable tokens. This can be too restrictive if the probability distribution is flat, or too broad if it's sharply peaked.CORRECT: Top-p (nucleus) sampling
Top-p (nucleus) sampling dynamically selects the number of tokens to consider. It chooses the smallest set of tokens whose cumulative probability is greater than or equal to 'p'. This allows for diversity while pruning the long tail of very unlikely tokens, addressing the developer's need.INCORRECT: Output length
Output length determines the maximum number of tokens in the generated text, not how the next token is chosen.
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
