AMAADOR ACADEMYCertificatesVerify a certificate

RAGE

Final check — Building RAG and LLM Applications

Answer the 43 questions. You need 80% or more to pass. You can retry as many times as you like.

Study the course →

Question 1Which problem does retrieval-augmented generation address best?
Question 2A 10-page FAQ rarely changes and a handful of staff ask about it each day. What is the simplest sound design?
Question 3Which RAG stages normally run offline, each time the documents change?
Question 4A fine-tuned assistant still quotes last year’s prices after the price list changed. What is the right fix?
Question 5A RAG assistant gives wrong answers. Logs show the passage with the correct rule never reached the prompt. Where should the team focus?
Question 6Why should a RAG answer display its sources?
Question 7A manager says a disclaimer on the chat window (“answers may be wrong”) is enough, so grounding checks are optional. What is the best response?
Question 8Why must documents and queries be embedded with the same model?
Question 9A team switches to a newer embedding model for new documents only. Search quality for questions drops. What is the right correction?
Question 10Which item is the most useful to store next to every vector so a later model migration stays safe?
Question 11Chunks of about 2000 tokens give vague matches and fill the prompt with irrelevant text. What is the best change?
Question 12What does contextual retrieval add to each chunk before it is embedded and indexed?
Question 13The corpus is in French and Arabic, and an English-focused embedding model returns poor matches. What should the team do?
Question 14In pgvector, which index type needs no training step and can be created on an empty table?
Question 15A multi-tenant assistant filters by tenant id. With a selective filter, a query for 10 results returns only 3. What is a sound remedy?
Question 16What does reciprocal rank fusion use to merge a keyword list and a vector list?
Question 17A team raises hnsw.ef_search from 40 to 200. What should they expect, and what must they do?
Question 18Why is a re-ranker applied only to a limited set of candidates, not to the whole index?
Question 19A question outside the corpus retrieves chunks with very low similarity, yet the assistant still writes a confident answer. What should be added?
Question 20Why label each source passage with an id such as [S3] in a grounded prompt?
Question 21Two retrieved documents give different notice periods: a 2022 policy and a 2025 policy. What should the prompt say?
Question 22When a model returns a tool call, who actually runs the tool?
Question 23The model returns JSON that is missing a required field for your order-lookup tool. What is the robust handling?
Question 24Which kind of tool needs the strictest controls such as limits and human approval?
Question 25Given a search tool, the model calls it 40 times on a vague question and the request times out. What should the team do?
Question 26Why evaluate retrieval and generation separately?
Question 27In RAG evaluation, what does a faithfulness (groundedness) score measure?
Question 28Recall at 5 is only 55% on the test set while answers are often wrong. What should the team improve first?
Question 29A judge model gives higher scores to longer answers even when they are less correct. What is the best response?
Question 30Besides answerable questions, which cases belong in a good test set?
Question 31A prompt change raises the average score, but a legal-question case now fails. What should the release process do?
Question 32What is indirect prompt injection?
Question 33The chat interface renders model output as Markdown and loads any image URL. A poisoned document makes the model output an image link that carries private data in its address. What control helps most?
Question 34Where should document permissions be enforced in a RAG application?
Question 35A developer puts an API key and internal server addresses in the system prompt “because users cannot see it”. What is the right call?
Question 36Which combination makes an LLM assistant especially dangerous?
Question 37A user exercises the right to erasure, and the source document is deleted. A week later the assistant still quotes it. What was missed?
Question 38Why put stable text at the start of a prompt and variable text at the end?
Question 39The model provider has an outage during working hours. Which prepared fallback keeps users productive?
Question 40Why track the 95th percentile of latency and not only the average?
Question 41After a model upgrade, complaints rise within hours. The team kept versions of the prompt, model and index. What is the best next step?
Question 42Which data belongs in a per-request trace for later debugging?
Question 43The nightly ingestion job has failed silently for three weeks, and the assistant quotes an outdated procedure. What monitoring was missing?