Building RAG and LLM Applications
Build a grounded LLM application: chunk and embed documents, retrieve and re-rank, write grounded prompts, use tools, evaluate it, secure it and run it in production.
What you will learn
- Explain why LLMs need retrieval, describe the RAG pipeline and choose between prompting, RAG, fine-tuning and long context.
- Prepare documents for retrieval: clean, chunk, add metadata and pick and version an embedding model.
- Design retrieval: choose a vector store and index, apply filters, combine keyword and vector search and add re-ranking.
- Write grounded prompts with citations and refusal rules, and use tool calls and structured outputs safely.
- Evaluate an LLM application with an offline test set, retrieval and answer metrics, LLM-as-judge and regression checks.
- Defend an LLM application against prompt injection, data leakage and excessive agency with layered guardrails.
- Plan cost, latency, monitoring and release for production, including caching, fallbacks and incident response.
Modules
- From LLM to RAG: the big picture
- Preparing documents: embeddings, chunking, metadata
- Retrieval: vector stores, hybrid search, re-ranking
- Grounded prompts, tool use and structured outputs
- Evaluating an LLM application
- Security and guardrails
- Going to production: cost, latency, operations
Course facts
- Estimated study time
- about 60 minutes
- Final assessment
- 43 questions · Pass mark 80%
- Available in
- English, Français, العربية
- Price
- Free. Optional verifiable certificate: $19
- Last updated
- Published by
- AMAADOR ACADEMY
Who this course is for
Developers, data and AI engineers, technical product owners and analysts who build or review LLM applications over company documents.
Prerequisites
Basic programming (Python or JavaScript), JSON and HTTP APIs, and some experience with a chat assistant. No machine-learning background needed.
Course content
1. From LLM to RAG: the big picture
A large language model (LLM) writes fluent text from patterns learned during training. That creates three practical problems for a company application: it does not know your private documents, its knowledge stops at a training cutoff date, and it can state wrong things with confidence.
Retrieval-augmented generation (RAG) fixes this by looking up relevant passages first and asking the model to answer from them. The model is no longer the source of truth; your documents are.
- Ingest — load PDFs, wiki pages, tickets; clean the text and keep the source link.
- Chunk — cut the text into passages of a useful size.
- Embed and index — turn each chunk into a vector and store it with its metadata.
- Retrieve — turn the question into a vector and fetch the closest chunks (often followed by re-ranking).
- Augment — place the best chunks in the prompt with clear instructions.
- Generate — the model answers from the chunks and cites them.
- Evaluate and monitor — measure quality offline and in production.
Steps 1 to 3 run offline whenever documents change. Steps 4 to 6 run for every question, so they set your latency and cost.
2. Preparing documents: embeddings, chunking, metadata
An embedding is a list of numbers (a vector) that represents the meaning of a text. Texts with similar meaning get vectors that are close together, so a question can find a passage that answers it even when the words differ. Closeness is measured with cosine similarity or dot product.
| Choice | What to check |
|---|---|
| Model | Languages covered (your French and Arabic documents!), maximum input length, quality on your own questions |
| Dimensions | More dimensions cost more storage and search time. For example OpenAI’s text-embedding-3-small returns 1536 numbers by default and text-embedding-3-large 3072; both accept a dimensions parameter to shorten the vector |
| Input limit | Those OpenAI models accept up to 8192 tokens per input, but a chunk should be far shorter |
| Hosting and privacy | Sending text to an external API is a data transfer; check contract and region |
Three rules cause most production incidents. First, use the same model for documents and queries: vectors from different models cannot be compared. Second, store the model name and version with every vector, because changing the model means re-embedding the whole corpus. Third, test on your own data: public leaderboards do not know your jargon or your Arabic dialect mix.
3. Retrieval: vector stores, hybrid search, re-ranking
A vector store keeps vectors and metadata and returns the nearest ones to a query vector. Scanning every vector gives exact results but gets slow as data grows, so most systems use an approximate nearest neighbour (ANN) index that trades a little recall for a lot of speed.
PostgreSQL with the pgvector extension is a common first choice because vectors live next to your normal tables. Its documentation describes two index types.
| Index | Strengths | Settings (documented defaults) |
|---|---|---|
| HNSW | Better speed and recall trade-off; no training step, so it can be created on an empty table; slower to build, more memory | m = 16, ef_construction = 64, query-time hnsw.ef_search = 40 |
| IVFFlat | Faster build, less memory, lower query performance; create it after loading data | lists (rows/1000 up to 1M rows), ivfflat.probes = 1 |
Operators matter: <=> is cosine distance, <-> is L2 distance and <#> is negative inner product. The index must be built for the operator you query with. The halfvec type stores half-precision numbers in about half the space.
4. Grounded prompts, tool use and structured outputs
Retrieval puts the right passages in front of the model. The prompt must then make it use only them. A grounded prompt has five parts, in a stable order.
- Role and scope — “You answer questions about HR leave policy for employees of X.”
- Rules — answer only from the sources; if they do not contain the answer, say so and suggest who to ask; never guess numbers or dates.
- Sources — each passage in a labelled block with an id, title and date, for example [S3].
- Output format — short answer first, then the source ids used; same language as the question.
- The question — last, kept separate from the sources.
Citations let a reader check the answer and let your code check it too: reject any citation id that was not in the prompt. Conflicts need a rule, such as “prefer the most recent document and mention the older one”. Multilingual apps should state the answer language explicitly, because French or Arabic questions can be answered from English sources.
5. Evaluating an LLM application
“It looks good in the demo” is not evaluation. You need a test set: real questions with known good answers, run automatically every time you change a chunk size, a model or a prompt.
- Collect 50 to 200 questions from real users, support tickets or search logs. Cover each document type, language and difficulty.
- Label each question with the passages that contain the answer and a short reference answer, written by someone who knows the domain.
- Add the hard cases — questions the corpus cannot answer, outdated policies, exact codes, vague follow-ups, and attacks (module 6).
- Freeze and version the set, and keep a separate small set you never tune on.
Evaluate the two halves separately, or you will not know what to fix.
| Half | Question | Metric examples |
|---|---|---|
| Retrieval | Did the right passage reach the prompt? | Hit rate and recall at k; mean reciprocal rank (MRR); context precision (are relevant chunks ranked above irrelevant ones?) and context recall (is everything needed there?) |
| Generation | Did the answer use it correctly? | Faithfulness (claims supported by the context), answer relevance, correct refusal on unanswerable questions |
6. Security and guardrails
An LLM cannot reliably tell instructions from data. Any text it reads can try to give it orders. That is prompt injection, ranked LLM01 in the OWASP Top 10 for LLM Applications 2025.
RAG makes indirect injection likely: you feed the model text you did not write.
Other OWASP 2025 items matter for RAG: LLM02 Sensitive Information Disclosure, LLM05 Improper Output Handling (the output is used by code or a browser without checks), LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, and LLM10 Unbounded Consumption.
Case: CVE-2025-32711, published in June 2025, records an “AI command injection” weakness in Microsoft 365 Copilot that allowed information disclosure over a network. Researchers described it as an attack where a crafted email, simply processed by the assistant, could make it reveal data. The lesson is general: if an assistant reads untrusted content and can reach private data, an outsider may steer it.
The risky combination: untrusted input, access to private data and a way to send data out. Remove at least one of the three.
7. Going to production: cost, latency, operations
In production, every question costs money and time. Measure both per request before you optimise anything.
Where the cost goes. Input tokens (instructions, retrieved chunks, history) are usually most of the bill, then output tokens. Each extra chunk, each tool loop and each re-ranking call adds up. Embedding and indexing is a one-off cost per document, plus a re-index whenever you change the embedding model.
| Lever | Effect | Watch out |
|---|---|---|
| Fewer, better chunks | Smaller prompt, lower cost and latency | Re-test recall |
| Prompt caching | Reuses a stable prompt prefix. In Anthropic’s documentation a 5-minute cache write costs 1.25 times the base input price and a cache hit about 0.1 times for most models (0.05 for some); a 1-hour option exists | Prompts shorter than the model’s minimum are not cached; any change to the prefix breaks the cache; check the current page for your model |
| Model routing | Small model for easy questions, larger for hard ones | Evaluate per group |
| Answer cache | Repeated questions cost nothing | Respect permissions and document updates |
| Batch jobs | Offline work (contextual chunk text, evaluation runs) can use cheaper batch options where the provider offers them | Slower turnaround |
The full lesson continues in the interactive course on this page.
Sources and references
- OWASP Top 10 for LLM Applications 2025
- Embeddings guide (OpenAI API documentation)
- Introducing Contextual Retrieval
- Prompt caching (Claude API documentation)
- pgvector: vector similarity search for Postgres
- Ragas documentation: RAG evaluation metrics
- AI Risk Management Framework and Generative AI Profile (NIST AI 600-1)
- CVE-2025-32711 record (AI command injection in a Microsoft 365 assistant)
Vendor-neutral educational course. Tools, model names, limits and prices change often: check the official documentation before you build or buy. It is not legal, security or compliance advice. Personal data in documents is governed by GDPR (EU), the French Loi Informatique et Libertés and Moroccan Law 09-08; involve your data protection officer.
All courses · AI & Data - Free courses, certificate from $7 | AMAADOR ACADEMY