Customize this training

Get a certificate — $16.67$5

Open Models and Running AI on Your Own Computer

Open Models and Running AI on Your Own Computer

A practical course on running AI models on your own computer: the difference between open-weight, open-source and closed models, how to read licences, how parameters and quantisation decide what your hardware can run, how to install and run a model safely with a local chat and API, and how to use and evaluate it honestly. Five modules, each with a worked example, and a 10-question final quiz. Tools are named neutrally and only as examples. Based on cited official pages. Not an accredited qualification and not legal advice.

What you will learn

  • Explain and apply: Distinguish open-weight, open-source and closed models, and weigh privacy, offline use, cost and control against the trade-offs of running locally.
  • Explain and apply: Read a model’s licence for commercial use, redistribution and usage restrictions, and explain the Open Source AI Definition at a high level.
  • Explain and apply: Estimate whether a model fits your hardware from parameters and quantisation, and explain the CPU versus GPU difference.
  • Explain and apply: Install and run a model safely: trusted sources, checksums, safe file formats, offline operation, and a local chat and API.
  • Explain and apply: Use a local model on small tasks and over your documents, evaluate it honestly, and keep it updated and secure.

Modules

  1. Open-weight, open-source and closed models
  2. Licences: read before you rely
  3. Hardware, parameters and quantisation
  4. Installing and running a model safely
  5. Using it well: tasks, retrieval, evaluation and security

Course facts

Estimated study time
about 75 minutes
Final assessment
10 questions · Pass mark 80%
Available in
English, Français, العربية
Price
Free. Optional verifiable certificate: $5
Last updated
Published by
AMAADOR ACADEMY

Who this course is for

Tech-curious professionals and developers who want to run open models on their own hardware, and need to choose, install and use them with care.

Prerequisites

Comfort with a terminal and basic file handling helps. No machine-learning background is required. No particular tool or brand is assumed.

Course content

1. Open-weight, open-source and closed models

A language model is, in practice, a very large file of numbers called weights, plus a small amount of code that knows how to run them. Three words are often used loosely. A closed model is reachable only through a provider’s service: you send text, you receive text, and you never hold the weights. An open-weight model is one whose trained weights you can download and run on your own machine, under whatever licence the publisher chose. A fully open-source model, in the stricter sense used in this course, would also come with the code and the information about training data needed to study and reproduce it. Many popular downloads are open-weight but not open-source in that stricter sense, and the difference matters for what you may legally do and for how much you can verify.

Why run a model locally at all? There are four common reasons. Privacy: the text you type stays on hardware you control, which can simplify some data-protection questions, although it does not remove your own duties. Offline use: once the files are on disk, the model works on a train, on a site with no signal, or in an isolated network. Cost: you pay for hardware and electricity instead of per request, which can be cheaper for steady, small workloads and more expensive for occasional use. Control: you choose the version, it does not change underneath you, and you can inspect and constrain how it is wrapped. None of these is automatic; each one depends on how you set things up, which is what the rest of the course covers.

2. Licences: read before you rely

Being able to download a file does not tell you what you may do with it. Every model published for download should come with a licence, usually in a file next to the weights or on the model page, and the licence is the document that decides. Licences for models vary widely. Some follow well-known software licences. Others are custom licences written by the publisher, with their own conditions. Two models that look similar in size and quality can allow very different things: one may let you use it commercially without conditions, another may limit commercial use above some scale, require you to display a notice, or forbid using its outputs to train other models. Never assume; read the text that actually ships with the version you downloaded.

Pay particular attention to usage restrictions. Some licences include an acceptable-use policy that lists prohibited purposes, and these can be binding conditions of the licence rather than friendly advice. Check whether redistribution is allowed if you plan to share a modified or compressed copy, whether attribution is required, and whether the licence covers the weights, the code, or both, since they can differ. If you work for an organisation, licence questions that affect products or customers belong with whoever handles legal review; this course can help you ask better questions but is not legal advice.

3. Hardware, parameters and quantisation

A model’s size is usually quoted in parameters, the individual numbers inside it: “7B” means roughly seven billion. Each parameter has to be stored, and the number of bytes per parameter depends on its precision. At 16 bits, which is 2 bytes, seven billion parameters need about 14 gigabytes just for the weights. That is approximate arithmetic, not a measurement (1 gigabyte is taken here as one billion bytes, and real files differ a little), and it is the first check to do before downloading anything. On top of the weights, the program needs extra memory for the context, meaning the text the model is currently reading and writing, and for its own working space. Longer contexts need more memory, so a model that loads may still run out of room on a long document.

Quantisation is the main trick that makes local use practical. It stores each weight with fewer bits, for example 4 bits instead of 16, which cuts the file to roughly a quarter of its size. By the same arithmetic, seven billion parameters at 4 bits come to roughly 3.5 gigabytes plus overhead. The cost is some loss of precision, which may or may not matter for your task; heavier compression tends to cost more quality, so test it. Quantised models are often distributed in formats such as GGUF, which llama.cpp reads, and tools such as Ollama package such files for you. Model cards usually list the available quantisation levels and their file sizes, which is better information than a rule of thumb.

4. Installing and running a model safely

A model download is software supply chain, and it deserves the care you would give any other executable dependency. Prefer the publisher’s official page or organisation account on a well-known hub such as Hugging Face, and be wary of near-identical names, re-uploads and unexplained mirrors. Many publishers list a checksum, a hash of the file, on the model page or alongside the files. After downloading, compute the hash of your copy with a standard tool and compare it with the published value: a match shows the file was not corrupted or swapped in transit, though it only proves integrity, not that the model is good or that its publisher is trustworthy.

The file format matters for security. Some model files are saved with Python’s pickle mechanism, and loading a pickle file can execute arbitrary code, so opening an untrusted one is like running an untrusted program. The safetensors format was designed to store only tensors, the arrays of numbers, without code, which removes that particular risk and also loads efficiently. GGUF, used by llama.cpp, is likewise a data format for model files. Prefer these formats, avoid loading untrusted pickle-based files, and treat any model that asks you to run custom code from its repository with the same suspicion as any script from the internet. A safe format does not make a model’s answers safe; it only means the file itself is not a program.

5. Using it well: tasks, retrieval, evaluation and security

Local models are at their best on small, well-defined tasks: summarising a page, extracting fields from text, rewriting for tone, classifying messages, drafting a first version for a human to edit. Give the model a narrow job, a clear instruction and, where possible, an example of the output you want. Smaller models tend to be less reliable on open-ended reasoning, long chains of steps or obscure facts, so do not ask them to be an oracle. They can also state wrong things fluently, which is why a person should check anything that matters before it is used.

To make a model answer from your own material, a common pattern is retrieval-augmented generation. Your documents are split into passages and indexed. For each question, the system finds the passages most relevant to it and places them in the prompt, so the model answers from that text instead of from memory. This does not retrain the model, and it keeps your files on your machine. It can still go wrong: retrieval can pick the wrong passage, and the model can ignore or misread it. Ask the model to quote the passage it used, and keep the source file open so you can check. Also note that text inside documents is untrusted input, and a document can contain instructions that try to steer the model.

The full lesson continues in the interactive course on this page.

Sources and references

Education only. It is not legal, financial or professional advice and not an accredited qualification. Licences, tools and defaults change; official sources are summarised as read on the access date, so check the current text and documentation. Figures in worked examples are illustrative.

All courses · AI & Data