What has to be true before a model ships

A model that scores well in a notebook and never reaches production is not a modelling failure. It is nearly always a data engineering failure that surfaced late, and the specific failures repeat across organisations with striking consistency.

The features have to be reproducible at serving time

The most common blocker: a feature computed from a table that only exists in the analytics warehouse, refreshed nightly, using a window that includes data not available at prediction time. The model works in training because it can see the future. In production it cannot, and accuracy collapses in a way that looks mysterious unless you go looking for leakage.

The fix is structural, not clever. Features get computed by one pipeline that serves both training and inference, or by two pipelines with a test that asserts they produce identical values for the same input. Anything less and the two drift, quietly.

Training data has to be reconstructible

Six months after deployment someone asks why the model made a particular decision. Answering means reconstructing the exact training set — which means the raw inputs, the transformation code and the feature definitions all have to be versioned together, not just the model weights.

If you cannot rebuild the training set from scratch, you do not have a reproducible model. You have an artefact.

In regulated settings this is not a nice-to-have. It is the thing a supervisor will ask for, and "we retrained since then" is not an answer.

Someone has to own degradation

Models decay. Input distributions shift, an upstream system changes a code list, a business process changes and the historical relationship stops holding. None of this raises an error — the model keeps returning confident predictions that are progressively less useful.

Production readiness means monitoring the inputs, not just the outputs: distribution checks on incoming features, alerts on null-rate and cardinality changes, and a scheduled review of prediction quality against outcomes once outcomes are known. And a named person who receives those alerts.

There has to be a rollback

The question "what do we do if this model starts behaving badly on a Friday afternoon" should have a boring answer: route traffic to the previous version, or to the deterministic rule the model replaced. If the answer involves a retraining run, the model is not deployed — it is merely running.

The honest sequencing

When an organisation asks for machine learning and the data layer underneath is not ready, the useful response is to say so and fix the layer first. The model built on unreliable inputs will be impressive in a demo and unusable in operations, and the credibility spent getting it approved does not come back.

Fixing the layer first is a slower announcement and a much faster route to something that survives contact with production.