Customize this training

Get a certificate — $34.29$12

Building RAG and LLM Applications

Building RAG and LLM Applications

Build a grounded LLM application: chunk and embed documents, retrieve and re-rank, write grounded prompts, use tools, evaluate it, secure it and run it in production.

What you will learn

  • Explain why LLMs need retrieval, describe the RAG pipeline and choose between prompting, RAG, fine-tuning and long context.
  • Prepare documents for retrieval: clean, chunk, add metadata and pick and version an embedding model.
  • Design retrieval: choose a vector store and index, apply filters, combine keyword and vector search and add re-ranking.
  • Write grounded prompts with citations and refusal rules, and use tool calls and structured outputs safely.
  • Evaluate an LLM application with an offline test set, retrieval and answer metrics, LLM-as-judge and regression checks.
  • Defend an LLM application against prompt injection, data leakage and excessive agency with layered guardrails.
  • Plan cost, latency, monitoring and release for production, including caching, fallbacks and incident response.

Modules

  1. From LLM to RAG: the big picture
  2. Preparing documents: embeddings, chunking, metadata
  3. Retrieval: vector stores, hybrid search, re-ranking
  4. Grounded prompts, tool use and structured outputs
  5. Evaluating an LLM application
  6. Security and guardrails
  7. Going to production: cost, latency, operations

Course facts

Estimated study time
about 60 minutes
Final assessment
43 questions · Pass mark 80%
Available in
English, Français, العربية
Price
Free. Optional verifiable certificate: $19
Last updated
Published by
AMAADOR ACADEMY

Who this course is for

Developers, data and AI engineers, technical product owners and analysts who build or review LLM applications over company documents.

Prerequisites

Basic programming (Python or JavaScript), JSON and HTTP APIs, and some experience with a chat assistant. No machine-learning background needed.

Course content

1. From LLM to RAG: the big picture

A large language model (LLM) writes fluent text from patterns learned during training. That creates three practical problems for a company application: it does not know your private documents, its knowledge stops at a training cutoff date, and it can state wrong things with confidence.

Retrieval-augmented generation (RAG) fixes this by looking up relevant passages first and asking the model to answer from them. The model is no longer the source of truth; your documents are.

  1. Ingest — load PDFs, wiki pages, tickets; clean the text and keep the source link.
  2. Chunk — cut the text into passages of a useful size.
  3. Embed and index — turn each chunk into a vector and store it with its metadata.
  4. Retrieve — turn the question into a vector and fetch the closest chunks (often followed by re-ranking).
  5. Augment — place the best chunks in the prompt with clear instructions.
  6. Generate — the model answers from the chunks and cites them.
  7. Evaluate and monitor — measure quality offline and in production.

Steps 1 to 3 run offline whenever documents change. Steps 4 to 6 run for every question, so they set your latency and cost.

2. Preparing documents: embeddings, chunking, metadata

An embedding is a list of numbers (a vector) that represents the meaning of a text. Texts with similar meaning get vectors that are close together, so a question can find a passage that answers it even when the words differ. Closeness is measured with cosine similarity or dot product.

ChoiceWhat to check
ModelLanguages covered (your French and Arabic documents!), maximum input length, quality on your own questions
DimensionsMore dimensions cost more storage and search time. For example OpenAI’s text-embedding-3-small returns 1536 numbers by default and text-embedding-3-large 3072; both accept a dimensions parameter to shorten the vector
Input limitThose OpenAI models accept up to 8192 tokens per input, but a chunk should be far shorter
Hosting and privacySending text to an external API is a data transfer; check contract and region

Three rules cause most production incidents. First, use the same model for documents and queries: vectors from different models cannot be compared. Second, store the model name and version with every vector, because changing the model means re-embedding the whole corpus. Third, test on your own data: public leaderboards do not know your jargon or your Arabic dialect mix.

3. Retrieval: vector stores, hybrid search, re-ranking

A vector store keeps vectors and metadata and returns the nearest ones to a query vector. Scanning every vector gives exact results but gets slow as data grows, so most systems use an approximate nearest neighbour (ANN) index that trades a little recall for a lot of speed.

PostgreSQL with the pgvector extension is a common first choice because vectors live next to your normal tables. Its documentation describes two index types.

IndexStrengthsSettings (documented defaults)
HNSWBetter speed and recall trade-off; no training step, so it can be created on an empty table; slower to build, more memorym = 16, ef_construction = 64, query-time hnsw.ef_search = 40
IVFFlatFaster build, less memory, lower query performance; create it after loading datalists (rows/1000 up to 1M rows), ivfflat.probes = 1

Operators matter: <=> is cosine distance, <-> is L2 distance and <#> is negative inner product. The index must be built for the operator you query with. The halfvec type stores half-precision numbers in about half the space.

4. Grounded prompts, tool use and structured outputs

Retrieval puts the right passages in front of the model. The prompt must then make it use only them. A grounded prompt has five parts, in a stable order.

  1. Role and scope — “You answer questions about HR leave policy for employees of X.”
  2. Rules — answer only from the sources; if they do not contain the answer, say so and suggest who to ask; never guess numbers or dates.
  3. Sources — each passage in a labelled block with an id, title and date, for example [S3].
  4. Output format — short answer first, then the source ids used; same language as the question.
  5. The question — last, kept separate from the sources.

Citations let a reader check the answer and let your code check it too: reject any citation id that was not in the prompt. Conflicts need a rule, such as “prefer the most recent document and mention the older one”. Multilingual apps should state the answer language explicitly, because French or Arabic questions can be answered from English sources.

5. Evaluating an LLM application

“It looks good in the demo” is not evaluation. You need a test set: real questions with known good answers, run automatically every time you change a chunk size, a model or a prompt.

  1. Collect 50 to 200 questions from real users, support tickets or search logs. Cover each document type, language and difficulty.
  2. Label each question with the passages that contain the answer and a short reference answer, written by someone who knows the domain.
  3. Add the hard cases — questions the corpus cannot answer, outdated policies, exact codes, vague follow-ups, and attacks (module 6).
  4. Freeze and version the set, and keep a separate small set you never tune on.

Evaluate the two halves separately, or you will not know what to fix.

HalfQuestionMetric examples
RetrievalDid the right passage reach the prompt?Hit rate and recall at k; mean reciprocal rank (MRR); context precision (are relevant chunks ranked above irrelevant ones?) and context recall (is everything needed there?)
GenerationDid the answer use it correctly?Faithfulness (claims supported by the context), answer relevance, correct refusal on unanswerable questions

6. Security and guardrails

An LLM cannot reliably tell instructions from data. Any text it reads can try to give it orders. That is prompt injection, ranked LLM01 in the OWASP Top 10 for LLM Applications 2025.

RAG makes indirect injection likely: you feed the model text you did not write.

Other OWASP 2025 items matter for RAG: LLM02 Sensitive Information Disclosure, LLM05 Improper Output Handling (the output is used by code or a browser without checks), LLM07 System Prompt Leakage, LLM08 Vector and Embedding Weaknesses, and LLM10 Unbounded Consumption.

Case: CVE-2025-32711, published in June 2025, records an “AI command injection” weakness in Microsoft 365 Copilot that allowed information disclosure over a network. Researchers described it as an attack where a crafted email, simply processed by the assistant, could make it reveal data. The lesson is general: if an assistant reads untrusted content and can reach private data, an outsider may steer it.

The risky combination: untrusted input, access to private data and a way to send data out. Remove at least one of the three.

7. Going to production: cost, latency, operations

In production, every question costs money and time. Measure both per request before you optimise anything.

Where the cost goes. Input tokens (instructions, retrieved chunks, history) are usually most of the bill, then output tokens. Each extra chunk, each tool loop and each re-ranking call adds up. Embedding and indexing is a one-off cost per document, plus a re-index whenever you change the embedding model.

LeverEffectWatch out
Fewer, better chunksSmaller prompt, lower cost and latencyRe-test recall
Prompt cachingReuses a stable prompt prefix. In Anthropic’s documentation a 5-minute cache write costs 1.25 times the base input price and a cache hit about 0.1 times for most models (0.05 for some); a 1-hour option existsPrompts shorter than the model’s minimum are not cached; any change to the prefix breaks the cache; check the current page for your model
Model routingSmall model for easy questions, larger for hard onesEvaluate per group
Answer cacheRepeated questions cost nothingRespect permissions and document updates
Batch jobsOffline work (contextual chunk text, evaluation runs) can use cheaper batch options where the provider offers themSlower turnaround

The full lesson continues in the interactive course on this page.

Sources and references

Vendor-neutral educational course. Tools, model names, limits and prices change often: check the official documentation before you build or buy. It is not legal, security or compliance advice. Personal data in documents is governed by GDPR (EU), the French Loi Informatique et Libertés and Moroccan Law 09-08; involve your data protection officer.

All courses · AI & Data - Free courses, certificate from $7 | AMAADOR ACADEMY