SRE, Observability and CI/CD Pipelines
Set SLOs and error budgets, instrument services with metrics, logs and traces, run on-call and postmortems, and design safe CI/CD pipelines with canary and blue-green releases.
What you will learn
- Define SLIs, SLOs and error budgets for a service and use the budget to decide between shipping features and fixing reliability.
- Distinguish monitoring from observability and instrument a service with metrics, logs and traces using OpenTelemetry.
- Design actionable alerts based on symptoms and burn rate, and organise a sustainable on-call rotation.
- Run an incident with clear roles and write a blameless postmortem with tracked follow-up actions.
- Design a CI/CD pipeline with fast feedback, secure credentials, and infrastructure as code reviewed like application code.
- Choose and run a deployment strategy (rolling, blue-green, canary) with automated checks and a tested rollback.
Modules
- SRE principles, SLIs, SLOs and error budgets
- Monitoring and observability with OpenTelemetry
- Alerting and on-call
- Incident management and postmortems
- CI/CD pipeline design and infrastructure as code
- Deployment strategies and release safety
Course facts
- Estimated study time
- about 60 minutes
- Final assessment
- 40 questions · Pass mark 75%
- Available in
- English, Français, العربية
- Price
- Free. Optional verifiable certificate: $19
- Last updated
- Published by
- AMAADOR ACADEMY
Who this course is for
DevOps, platform and cloud engineers, developers who carry a pager, system administrators moving into SRE, and team leads who own service reliability.
Prerequisites
Basic Linux command line, Git, and a general idea of how a web application and an API work. Prior experience with a cloud platform or containers helps but is not required.
Course content
1. SRE principles, SLIs, SLOs and error budgets
Site Reliability Engineering (SRE) is the practice of running production systems with software engineering methods. It started at Google, whose free SRE book describes it. The core idea is simple: reliability is a feature, it has a cost, and it must be measured and traded against other work.
SRE and DevOps share goals (shared ownership, automation, fast delivery). SRE adds a precise tool: numeric reliability targets that both developers and operators accept.
The four habits of an SRE team.
Why not aim for 100%? Users cannot tell the difference between 99.99% and 100% because their phone, Wi-Fi and ISP are less reliable than that. Each extra “nine” costs far more engineering time and slows releases. A target of 100% also means you may never change anything.
Toil is manual, repetitive, automatable work that grows with the service and has no lasting value, for example restarting a service by hand every night. The Google SRE book suggests keeping toil below about half of an engineer’s time, so the rest goes to engineering that removes it.
2. Monitoring and observability with OpenTelemetry
Monitoring watches for known failure modes: you decide in advance what to measure and when to alert (“is CPU above 90%?”). Observability is the ability to ask new questions about a system from the data it emits, without shipping new code (“why are only users in one region slow?”). You need both.
| Signal | What it is | Best for | Watch out for |
|---|---|---|---|
| Metrics | Numbers aggregated over time | Dashboards, SLOs, alerts | High-cardinality labels (user ID) blow up cost |
| Logs | Timestamped event records | Detail on one event | Unstructured text, secrets and personal data in logs |
| Traces | One request followed across services, as spans | Finding which hop is slow | Sampling choices hide rare problems |
A practical starting point for services is the RED method: Rate of requests, Errors, Duration. For resources (CPU, disks, queues) use USE: Utilisation, Saturation, Errors. Google’s “four golden signals” are latency, traffic, errors and saturation.
Do
- Write logs as structured JSON with a trace ID.
- Alert from metrics tied to user impact.
- Keep metric labels to small sets of values.
3. Alerting and on-call
An alert that wakes someone at 3 a.m. must be urgent, actionable and caused by real user impact. The Prometheus documentation and the Google SRE book give the same advice: alert on symptoms (users see errors or slowness), not on every possible cause (a disk at 80%).
| Alert type | Example | Goes to |
|---|---|---|
| Page (wake a human) | Checkout error rate burning the monthly budget fast | Pager |
| Ticket (next working day) | Certificate expires in 14 days | Queue |
| Dashboard only | CPU at 75% | Nobody |
Burn-rate alerts link alerts to the SLO. The burn rate is how fast you consume the error budget: 1 means you will use exactly the whole budget by the end of the window. The SRE Workbook suggests multi-window alerts, for example a page when the burn rate is at least 14.4 over 1 hour (about 2% of a 30-day budget gone) and a page at 6 over 6 hours, plus tickets for slow burns. Checking a long and a short window together avoids both slow detection and false alarms.
4. Incident management and postmortems
During an incident, the goal is to restore service first, then find the root cause later. Chaos comes from unclear roles, so assign them at the start. Google’s SRE book and PagerDuty’s public response guide describe the same structure.
| Role | Job |
|---|---|
| Incident commander (IC) | Owns the response, makes decisions, delegates. Does not debug. |
| Operations lead | Works on the technical fix with responders. |
| Communications lead | Updates stakeholders and the status page on a fixed rhythm. |
| Scribe | Writes a timestamped log of actions and decisions. |
- Declare — open an incident channel, set a severity, name the IC.
- Mitigate — roll back, fail over, disable the feature flag, add capacity. A quick mitigation beats an elegant fix.
- Communicate — post updates every 15 to 30 minutes even if the update is “no change”.
- Resolve — confirm the SLIs are back to normal, then stand down.
- Hand off — schedule the postmortem and assign an owner.
Define severity levels by impact, not by cause. For example: SEV1 means most users cannot use a core function; SEV2 means a core function is degraded for many users; SEV3 means a minor issue with a workaround. Each level states who is paged and how often updates go out.
5. CI/CD pipeline design and infrastructure as code
Continuous integration (CI) merges small changes often and verifies each one automatically. Continuous delivery keeps every passing build ready to release; continuous deployment releases it automatically. The aim is fast, reliable feedback.
- Commit and build — compile once and produce one immutable artifact (for example a container image tagged with the commit SHA).
- Fast tests — lint, unit tests, static analysis in minutes.
- Security checks — dependency and secret scanning, image scanning, SBOM. NIST SP 800-218 (SSDF) describes the practices.
- Deploy to staging — the same artifact, then integration and smoke tests.
- Deploy to production — progressive, with an approval or automatic gates.
Build once, deploy many. Never rebuild for production: what you tested must be the exact artifact you ship. Keep configuration outside the artifact.
Measure the pipeline with the DORA metrics. DORA currently groups five: throughput (change lead time, deployment frequency, failed deployment recovery time) and instability (change fail rate, deployment rework rate).
| Metric | Question it answers |
|---|---|
| Change lead time | How long from commit to production? |
| Deployment frequency | How often do we ship? |
| Failed deployment recovery time | How fast do we recover from a bad release? |
| Change fail rate | What share of releases need a fix or rollback? |
| Deployment rework rate | How many extra, unplanned deployments do we make to fix problems? |
6. Deployment strategies and release safety
Every deployment strategy balances speed, risk, cost and rollback time. Pick by what a failure would cost.
| Strategy | How it works | Strength | Weakness |
|---|---|---|---|
| Recreate | Stop the old version, start the new one | Simple | Downtime |
| Rolling | Replace instances a few at a time (the Kubernetes Deployment default) | No extra capacity, no downtime | Two versions run together; slower rollback |
| Blue-green | Run the new version beside the old and switch all traffic at once | Instant rollback by switching back | Double capacity during the switch; database changes need care |
| Canary | Send a small share (for example 5%) to the new version, then raise it | Limits the blast radius; real traffic validates the release | Needs good metrics and traffic control |
Tools such as Argo Rollouts and Flagger add canary and blue-green to Kubernetes. Argo Rollouts can compare metrics from a provider like Prometheus and promote or abort automatically.
- Deploy to 5% of traffic.
- Compare error rate and latency of canary versus stable for a set period.
- Promote to 25%, 50%, 100% only while SLIs stay within limits.
- Abort and roll back automatically if a threshold is crossed.
The full lesson continues in the interactive course on this page.
Sources and references
- Site Reliability Engineering (free online book)
- The Site Reliability Workbook: Implementing SLOs and Alerting on SLOs
- OpenTelemetry signals: traces, metrics, logs
- Prometheus alerting best practices
- DORA software delivery metrics guide
- OpenID Connect in GitHub Actions deployments
- Argo Rollouts: progressive delivery concepts
- NIST SP 800-218: Secure Software Development Framework (SSDF)
Professional training course. It is not affiliated with or endorsed by Google, the CNCF, the Linux Foundation, GitHub or any tool vendor named here; product names are trademarks of their owners. Tool features and versions change quickly: check the official documentation before applying anything in production, and test changes outside production first.
All courses · DevOps & Platform Engineering — Free certified training