Glossary
The vocabulary of running production systems: incident response, reliability engineering, observability, and the AI systems now doing part of the work.
Some terms, particularly in AI and automation, are emerging, and usage varies across vendors and practitioners. These definitions describe how the terms are commonly used in modern production engineering rather than prescribing a formal industry standard.
Agentic AI
Agentic AI describes systems that pursue a goal across multiple steps, choosing their own actions and tools along the way, rather than producing a...
AI & automationAgentic Mission Control
In this glossary, agentic mission control refers to the operating surface where humans supervise fleets of AI agents working on infrastructure and...
AI & automationAI for On-Call
In this glossary, AI for on-call refers to systems that carry part of the on-call burden: triaging pages, gathering context before a human reads the...
AI & automationAI for Production Systems
In this glossary, AI for production systems refers to the application of machine learning and language models to the operation of live, customer-facing...
AI & automationAI SRE
In this glossary, an AI SRE refers to an autonomous system that performs site reliability engineering work: investigating incidents, correlating...
AI & automationAIOps
AIOps (Artificial Intelligence for IT Operations) is the practice of applying machine learning and statistical analysis to operational data so that...
AI & automationAlert Intelligence
In this glossary, alert intelligence refers to the layer that decides which alerts deserve human attention.
AI & automationAutomated Incident Response
Automated incident response is the execution of detection, diagnosis, and remediation steps without waiting for human action.
AI & automationAutonomous IT Operations
Autonomous IT operations describes infrastructure that detects, diagnoses, and corrects its own problems without human involvement in the routine cases.
AI & automationCausal Search Engine
In this glossary, a causal search engine refers to a system that searches for the cause of a failure rather than for documents matching keywords.
AI & automationCode Resilience
In this glossary, code resilience refers to the property of software that continues to function acceptably when its dependencies fail, its inputs are...
AI & automationContext Engineering
Context engineering is the practice of assembling the right information for an AI system at the right moment.
AI & automationMulti-Model Orchestration
Multi-model orchestration is the coordination of several AI models within one workflow, routing each step to the model best suited for it.
AI & automationVibe Coding
Vibe coding is writing software by describing intent to an AI model and iterating on what it produces, rather than authoring the implementation directly.
AI & automationVibe Debugging
In this glossary, vibe debugging refers to diagnosing a failure by describing the symptoms to an AI system and working through its hypotheses, rather...
AI Incident Management Platform
In this glossary, an AI incident management platform refers to a system that coordinates the full lifecycle of an incident, from detection through...
Incident managementAlert Fatigue
Alert fatigue is the desensitization that follows sustained exposure to alerts that do not require action.
Incident managementAlert Investigations
An alert investigation is the work between an alert firing and a responder understanding what is happening: confirming the alert is real, establishing...
Incident managementAlert Noise
Alert noise is the portion of alert volume that carries no actionable information: duplicates of a single condition, alerts on transient blips that...
Incident managementBlast Radius
Blast radius is the extent of what a failure or change can affect.
Incident managementEscalation
Escalation is the transfer of an incident to additional or more senior responders when the current responder cannot resolve it within an expected window.
Incident managementIncident
An incident is an unplanned event that disrupts or degrades a service, threatens its reliability, security, or data integrity, or otherwise requires...
Incident managementOn-Call
On-call is the rotation that guarantees a human is available to respond to production problems at any hour.
Incident managementPager Fatigue
Pager fatigue is the cumulative exhaustion produced by carrying on-call responsibility: interrupted sleep, disrupted personal time, and the constant...
Incident managementPostmortem
A postmortem is the structured review conducted after an incident to establish what happened, why the system allowed it, and what should change.
Incident managementProactive Incident Management
Proactive incident management is the practice of finding and correcting conditions that will cause incidents before they do.
Incident managementRoot Cause Analysis (RCA)
Root cause analysis is the systematic investigation of why a failure occurred, tracing from the observed symptom back to the conditions that produced it.
Incident managementRunbook
A runbook is a documented procedure for handling a specific operational situation: what to check, in what order, and what to do based on what you find.
Incident managementSeverity
Severity is the classification of how serious an incident is, which determines response urgency, who gets involved, and how widely it is communicated.
Availability
Availability is the proportion of time or requests for which a system is operational and able to serve, expressed as a percentage over a defined window.
Reliability & SLOsDORA Metrics
DORA metrics are five measures of software-delivery performance identified by the DevOps Research and Assessment program: change lead time, deployment...
Reliability & SLOsDowntime
Downtime is the period during which a system is unavailable to its users.
Reliability & SLOsError Budget
An error budget is the amount of unreliability a service is permitted over a period, derived directly from its SLO.
Reliability & SLOsMTBF
Mean time between failures is the average operating time between one failure and the next for a repairable system.
Reliability & SLOsMTTA
Mean time to acknowledge is the average time between an alert firing and a responder confirming they have seen it and are engaging.
Reliability & SLOsMTTD
Mean time to detect is the average time between a failure beginning and someone or something noticing it.
Reliability & SLOsMTTF
Mean time to failure is the average operating time before a non-repairable component fails permanently.
Reliability & SLOsMTTM
Mean time to mitigate is the average time between detecting a problem and reducing its user impact to an acceptable level.
Reliability & SLOsMTTR
Mean time to recovery is the average time from the start of an incident to the restoration of normal service.
Reliability & SLOsNines of Availability
Nines of availability is shorthand for expressing reliability targets by counting the nines in a percentage.
Reliability & SLOsSLA
A service level agreement is an explicit or implicit agreement with users that defines expected service levels and the consequences associated with...
Reliability & SLOsSLI
A service level indicator is the actual measurement of some aspect of service quality: the number that gets compared against a target.
Reliability & SLOsSLO
A service level objective is a target value or range of values for a service level measured by an SLI.
Reliability & SLOsSoftware Reliability
Software reliability is the probability that software performs its intended function correctly for a specified period under specified conditions.
Reliability & SLOsUptime
Uptime is the duration or proportion of time a system has been operational.
Golden Signals
The four golden signals are latency, traffic, errors, and saturation: the minimum set of measurements that describe the health of a user facing service.
ObservabilityMELT
MELT is a mnemonic for metrics, events, logs, and traces, four commonly used forms of telemetry in observability systems.
ObservabilityObservability
Observability is the property of a system that lets you understand its internal state from its external outputs.
ObservabilityOpenTelemetry (OTel)
OpenTelemetry is a vendor-neutral, open-source observability framework and toolkit for generating, collecting, processing, and exporting telemetry such...
Blue-Green Deployment
Blue-green deployment runs two identical production environments and switches traffic between them.
DeploymentCanary Deployment
A canary deployment releases a new version to a small fraction of traffic first, monitors it against the existing version, and expands only if the...
DeploymentChange Management
Change management is the process governing how modifications reach production: what review is required, how changes are tracked, and how they are reversed.
DeploymentCI/CD
CI/CD is continuous integration and continuous delivery or deployment: the automated pipeline that takes a code change from commit through build, test,...
DeploymentRollback
A rollback returns a system to a previous known good state after a change causes problems.
Bring Your Own Cloud (BYOC)
Bring your own cloud is a deployment model in which some or all of a vendor's software runs in cloud infrastructure controlled or paid for by the customer.
SRE conceptsChaos Engineering
Chaos engineering is the practice of deliberately injecting failures into a system to discover weaknesses before they surface on their own.
SRE conceptsCircuit Breaker
A circuit breaker stops calls to a failing dependency after a threshold of failures, returning an error immediately instead of waiting for a timeout.
SRE conceptsDay 2 Operations
Day 2 operations is everything that happens after a system is live: upgrades, scaling, patching, backup verification, certificate rotation, and...
SRE conceptsDebugging
Debugging is the process of finding why software behaves incorrectly.
SRE conceptsDevOps
DevOps is a set of practices that shortens the path from code change to running software by removing the separation between development and operations.
SRE conceptsGraceful Degradation
Graceful degradation is the design property where a system reduces functionality under failure or load instead of failing completely.
SRE conceptsKubernetes (K8s)
Kubernetes is an open-source system for automating the deployment, scaling, and management of containerized applications.
SRE conceptsOOMKilled Errors
OOMKilled is a container termination reason indicating that the Linux out-of-memory killer terminated a process because memory could not be allocated.
SRE conceptsPod Pending State
A Pod in the Pending phase has been accepted by the Kubernetes cluster, but one or more of its containers has not yet been set up and made ready to run.
SRE conceptsProduction Readiness
Production readiness is the set of criteria a service must meet before it carries real traffic: monitoring, alerting, documented ownership, tested...
SRE conceptsProduction Systems
Production systems are live environments that perform real business, customer, or operational workloads.
SRE conceptsSite Reliability Engineering (SRE)
Site reliability engineering is the discipline of applying software engineering methods to operations problems.
SRE conceptsToil
In SRE, toil is work tied to running a production service that tends to be manual, repetitive, automatable, tactical, lacking enduring value, and...