Customize this training

Get a certificate — $34.29$12

MLOps and Model Deployment Fundamentals

MLOps and Model Deployment Fundamentals

Take a machine learning model from notebook to production: track experiments, version data and models, package and serve them, automate CI/CD, monitor drift and document for governance.

What you will learn

  • Describe the ML lifecycle and the three MLOps maturity levels, and explain why models fail in production.
  • Track experiments and make a training run reproducible with logged parameters, metrics, code version and pinned environment.
  • Version data and models, and use a model registry with aliases and lineage to promote a model safely.
  • Package a model with a signature and a pinned environment, and choose between batch and real-time serving and a rollout strategy.
  • Design a CI/CD pipeline for ML with data checks, model tests, quality gates and automated rollback.
  • Monitor a deployed model for drift and performance, and decide when and how to retrain, using a feature store to avoid skew.
  • Document models and operate them under governance: model cards, audit trail, access control, incident runbooks and regulatory duties.

Modules

  1. From notebook to production
  2. Experiment tracking and reproducibility
  3. Versioning data and models
  4. Packaging and serving models
  5. CI/CD and pipelines for ML
  6. Monitoring, drift and retraining
  7. Governance, documentation and readiness

Course facts

Estimated study time
about 60 minutes
Final assessment
42 questions · Pass mark 80%
Available in
English, Français, العربية
Price
Free. Optional verifiable certificate: $19
Last updated
Published by
AMAADOR ACADEMY

Who this course is for

Data scientists, ML and software engineers, data engineers, DevOps staff and technical team leads who need to run models in production. Also useful for analysts moving into an MLOps role.

Prerequisites

Basic Python, Git and command-line skills, and a first idea of how a model is trained and evaluated. No cloud account is needed to follow the course.

Course content

1. From notebook to production

A model in a notebook is a prototype. A model in production is a service: it must run every day, on fresh data, for real users, and someone must answer when it breaks. MLOps (machine learning operations) is the set of practices that make this reliable by applying DevOps ideas to code, data and models together.

The ML lifecycle is a loop, not a line: monitoring feeds the next round of training.

In their well-known paper on hidden technical debt in ML systems, Sculley and colleagues (Google, NeurIPS 2015) observed that the model code is only a small box inside a much larger system: data collection, feature extraction, configuration, serving, monitoring. Most production problems come from that surrounding system, not from the algorithm.

Typical reasons models fail after launch:

  • Data changes. Customers, sensors or prices change, and the input no longer looks like the training set.
  • Training-serving skew. Features are computed one way in training and another way in production.
  • No reproducibility. Nobody can rebuild the model that is live.
  • No monitoring. Errors are found by customers, weeks later.

2. Experiment tracking and reproducibility

Data science is trial and error. Without a record, you cannot say which run was best or why. An experiment tracker stores, for each run, what you changed and what happened. MLflow Tracking is one of the most widely used open-source options; Weights & Biases, Neptune and cloud platforms offer similar features.

MLflow organises work into experiments (a project or question) and runs (one training attempt). In Python the core calls are:

  1. mlflow.set_experiment("churn") — choose where runs are stored.
  2. mlflow.start_run() — open a run, usually in a with block.
  3. mlflow.log_param("max_depth", 6) — record an input setting.
  4. mlflow.log_metric("auc", 0.91) — record a result, optionally per step.
  5. mlflow.log_artifact("confusion.png") — attach a file such as a plot or report.

Calling mlflow.autolog() before training logs parameters, metrics and the model automatically for popular libraries such as scikit-learn, XGBoost and PyTorch Lightning. Start the UI with mlflow ui to sort and compare runs side by side. In a team, run a shared tracking server with a database and artifact store rather than local folders.

Log thisWhy
Parameters and config fileTo repeat the run
Git commit hash and dataset version tagTo find the exact code and data
Metrics on train, validation and testTo compare runs and spot overfitting
Plots, feature list, model fileTo inspect results without re-running
Run tags: owner, purposeTo search later

3. Versioning data and models

Git is built for small text files. Datasets and model files are large and binary, so Git alone cannot track them well. DVC (Data Version Control) solves this: it keeps a small text pointer in Git and stores the real data in a remote such as S3, Azure Blob, Google Cloud Storage, SSH or a shared drive.

  1. dvc init — enable DVC inside a Git repository.
  2. dvc add data/raw.csv — DVC creates data/raw.csv.dvc (a pointer with a content hash) and adds the data file to .gitignore.
  3. git add data/raw.csv.dvc .gitignore then commit — the data version is now tied to a code commit.
  4. dvc remote add -d storage s3://bucket/path then dvc push — upload the data.
  5. dvc pull — on another machine, after git checkout, fetch the matching data.

DVC can also describe a pipeline in a dvc.yaml file: each stage lists its command, dependencies (code, data, parameters) and outputs. The command dvc repro reruns only the stages whose inputs changed, and dvc exp run manages experiments. Parameters live in params.yaml, and dvc metrics diff compares results between commits.

4. Packaging and serving models

A trained model object is not yet deployable. A package bundles the model with everything needed to run it: the serialised weights, the preprocessing, the expected inputs and outputs, and the software environment. MLflow Models does this with a model flavor (scikit-learn, PyTorch, XGBoost, or the generic python_function flavor), a signature and a requirements list, all stored with the model.

The signature declares the column names and types the model expects and returns. You create it with infer_signature(X_train, predictions) and pass it when logging the model. Serving can then reject a request with a missing column before it reaches the model.

FormatNotes
pickle / joblibEasy for Python, but loading a file from an untrusted source can execute code. Load only files you produced
ONNXFramework-neutral; runs with ONNX Runtime in many languages
safetensorsSafe, fast weight storage for deep learning models
MLflow ModelFolder with model, signature, environment files

Package the preprocessing with the model, for example in a scikit-learn Pipeline. Otherwise the serving code will reimplement it slightly differently and create skew.

5. CI/CD and pipelines for ML

In normal software, continuous integration (CI) runs tests on every change. In ML, a change can be new code, new data or a new model, so CI must test all three. A CI service such as GitHub Actions, GitLab CI or Jenkins runs the checks automatically on each pull request, and the merge is blocked if they fail.

WhatExample checksTools
CodeUnit tests for feature functions, linting, type checkspytest, ruff, mypy
DataSchema, types, value ranges, null rates, duplicates, row countsGreat Expectations, pandera, TensorFlow Data Validation
ModelMetric at least as good as the baseline, no slice falls below a floor, prediction on fixed examples, latency budgetpytest with MLflow, custom scripts
PackageImage builds, loads and predicts in a clean container; dependency scanDocker, pip-audit, Trivy

Three model tests catch many bugs cheaply:

  • Quality gate — fail the build if the validation metric drops below the current champion by more than a set margin.
  • Slice tests — check the metric for each important group (region, device, new vs old customers), because an average can hide a failing group.
  • Behaviour tests — a prediction must not change when an irrelevant detail changes (invariance), and must move the right way when a key input changes (directional test).

6. Monitoring, drift and retraining

A deployed model is never finished. The world changes, so quality decays. Monitoring answers: is the service healthy, is the input still normal, are the outputs sensible and, when the truth arrives, are the predictions still right?

LayerExample signalsSpeed
SystemLatency p95, errors, CPU, memory, queue lengthSeconds
Data qualityMissing values, new categories, out-of-range values, schema changesMinutes to hours
Data and prediction driftInput and output distributions versus a reference windowHours to days
Model qualityAccuracy, AUC, error against real outcomesDelayed by label arrival
BusinessConversion, loss rate, complaintsDays to weeks

Distinguish the kinds of change:

  • Data drift (covariate shift) — the inputs change, for example a new customer segment.
  • Concept drift — the relationship between inputs and the target changes, for example fraud tactics evolve.
  • Training-serving skew — production features are computed differently from training.

To detect drift, compare a recent window of production data with a reference, often the training data. Common tests are the Population Stability Index (PSI), whose widely used rule of thumb treats under 0.1 as stable and over 0.25 as a major shift, and the Kolmogorov-Smirnov test for numeric features. Open-source libraries such as Evidently produce drift and data-quality reports and tests. A drift alert means look, not automatically retrain: check whether quality really fell.

7. Governance, documentation and readiness

Once a model affects customers, money or safety, people will ask: what is it for, how was it built, how well does it work for whom, and who is responsible? Governance means having written, findable answers.

A model card (Mitchell et al., 2019) is a short document that travels with the model. Datasheets for datasets (Gebru et al.) do the same for data.

Model card sectionContent
Intended useWhat it is for, and what it must not be used for
DataSources, time span, known gaps, personal data handling
PerformanceMetrics overall and per slice, with the test set named
Limits and risksKnown failure modes, bias findings, fallback behaviour
OperationsVersion, owner, monitoring, retraining plan, rollback

Keep an audit trail: who trained, approved and promoted each version, on which data, and which version produced which prediction. Log predictions with the model version and a request ID, within your privacy rules. Use role-based access so that not everyone can change production aliases or read raw personal data.

Frameworks to know:

The full lesson continues in the interactive course on this page.

Sources and references

Educational course on engineering practice. Tools and versions change quickly: check the official documentation of each tool before you adopt it. The regulatory points (EU AI Act, GDPR, Moroccan Law 09-08) are an overview, not legal advice. Ask your legal or compliance team about your own obligations.

All courses · AI & Data - Free courses, certificate from $7 | AMAADOR ACADEMY