Databricks Certified Generative AI Engineer Associate · Free Practice Question Medium
Question 15
- A Add a classifier for user queries that predicts which book will best contain the answer. Use this to filter retrieval.
- B Pass known questions and best answers to an LLM and instruct it to identify optimal token counts. Use a summary statistic (mean, median, etc.) to determine chunk sizes.
- C Create an LLM-as-a-judge metric to assess how well previous questions are answered by specific chunks. Use this metric to optimize chunking parameters.
- D Change embedding models and compare performance.
- E Choose an appropriate evaluation metric (e.g., recall or NDCG) and test chunking strategies such as splitting chunks by paragraphs or chapters. Select the strategy with the highest performance metric.
Reveal correct answers
Correct answers: C, E
Explanation
A systematic evaluation using metrics like recall and NDCG, combined with LLM-as-a-judge metrics, ensures the best chunking strategy is selected for optimal performance.A. Incorrect: While helpful for narrowing retrieval, a classifier does not optimize the chunking strategy itself.
B. Incorrect: LLM-driven token suggestions lack a structured approach for robust chunk optimization.
C. Correct: An LLM-as-a-judge approach allows the engineer to systematically evaluate and refine chunk relevance and sizes.
D. Incorrect: Experimenting with embedding models improves semantic alignment but does not directly optimize chunking strategies.
E. Correct: Using evaluation metrics provides a structured way to identify the most effective chunking strategies.
Discussion
Think the marked answer is wrong, or have a better explanation? Share it below — comments appear after review.
