Start here
MLOps in plain language
A model file is only one part of the deployed system. Changes to input preparation or data can alter predictions even if the model itself is unchanged.
Operational health and predictive quality are separate. A fast, available endpoint can still produce poor predictions after its input population changes.
How it works
A model release needs a feedback path
Operate the whole ML system
- Version + evaluate
Record data, code, model and acceptance tests.
- Release
Deploy the approved combination with a recovery plan.
- Monitor
Inspect service health, input changes and outcome quality.
Investigate a change before deciding whether to retrain, repair data or revert a release.
Make a reproducible candidate
Record data selection, preprocessing, dependencies and training configuration.
Compare before release
Evaluate against the current model and relevant baselines using the same test conditions.
Monitor and respond
Define who investigates alerts and how a prior working version can be restored.
Illustrative example · Fictional data
A fictional routing model encounters a new product
- Input
- A support classifier begins receiving messages about a newly launched product that was absent from its evaluation set.
- Process
- Inspect misroutes, new vocabulary and labelling quality. Create representative test cases before updating the model.
- Expected output
- A reviewed change with evidence that new-product routing improves without unacceptable regressions elsewhere.
- How to check it
- Do not retrain on the model’s own guesses as if they were verified labels. Keep a reliable evaluation set separate.
Where it helps
- Maintaining deployed prediction services.
- Tracing a result to the model and processing version.
- Managing model updates and recovery.
DevOps, MLOps and LLMOps
- DevOps
- Connects software delivery with operation.
- MLOps
- Adds model, data and predictive-evaluation concerns.
- LLMOps
- Applies operational practices to language-model applications, including prompts, retrieval and generated-output evaluation.
Experiments, versions and registries
Experiment tracking records the data reference, code version, configuration, metrics and artefacts for a run. Data versioning identifies the inputs used. A model registry records model versions and their status or approval evidence. These responsibilities connect a production prediction to the work that produced it.
MLflow provides tracking and registry capabilities; DVC is one approach to data and pipeline versioning. Cloud ML platforms can combine managed training, deployment and workflow features. Select tools around traceability and operational needs, not a requirement to install every platform.
A minimum useful setup for a small team
Keep reproducible code and dependency versions, identifiable training data, a saved model, a held-out evaluation and a repeatable deployment. Name the owner, preserve a previous working version and define how an incident is detected and handled. Add automation as the number of models and changes grows.
Monitor input changes, latency, failures and outcomes when reliable labels arrive. A drift alert is evidence to investigate, not proof accuracy has fallen. Retraining should pass evaluation and release checks before replacing a working model; automatically retraining on new data can reproduce bad labels or a transient incident.
Related reading and learning pathways
Common pitfalls
- Input drift and accuracy loss are related but not identical.
- Outcome labels may arrive long after predictions.
- An unversioned preprocessing change can break reproducibility.
Put the idea to work
Try it yourself
Write an incident record that identifies a prediction’s input version, preprocessing version, model and observed outcome. Mark anything you could not currently reconstruct.
Common questions
Does MLOps require automatic retraining?
No. Retraining should follow justified triggers and evaluation, not merely an alert. Retraining can introduce new errors, so compare the new model against the current one on held-out data before release, and keep the ability to roll back.
Is an available endpoint enough?
No. Measure service health and prediction quality separately. An endpoint can respond quickly while its predictions drift because the input data has changed. Monitor latency and errors alongside accuracy, data drift and business outcomes. Alert on quality signals, not only on uptime.
Your next step
Continue learning
Connected concepts
Sources and further reading
Examples on this page are teaching exercises using fictional information. They are not client case studies or measured outcomes.