Skip to content
ml-service-blueprint

MIT · Python 3.11–3.12 · GitHub template

Training is the easy half.

A template repository for everything that happens after model.fit() — immutable artifacts, a registry with real rollback, schema enforcement at the API boundary, structured observability, and a container CI actually builds. The example model is a small tabular classifier on purpose, so nothing distracts from the infrastructure.

Use this template Read the golden path
Tests
172 passing
CI jobs
6 green
Typing
mypy strict
Frontend
None. On purpose.
make golden-path 15 passed 
Data            PASS  dataset materialises
Training        PASS  pipeline trains a model
Validation      PASS  metrics clear the gates
Artifact        PASS  contract complete, round-trips
                PASS  tampered artifacts rejected
Registry        PASS  promote / rollback / history
Service         PASS  all endpoints respond
Observability   PASS  logs, request ids, metrics
Drift hooks     PASS  sink, detector, fail-open
CI checks       PASS  lint, typecheck, tests
Container       SKIP  docker not installed here

15 passed, 0 failed, 1 skipped

That SKIP is the point. A step whose prerequisites are missing is reported as skipped, with the reason — never quietly counted as a pass. The container is verified separately, in CI, on a runner that has Docker.

Clone & run

One command runs every stage.

A GitHub template, not a package. Python 3.11 or 3.12. make golden-path is not a smoke test with a happy ending — it executes each stage, verifies the result, and prints PASS, FAIL, or SKIP with the reason.

  1. Use the template

    Create your own repository from it, then clone that.

    gh repo create my-service --template Gariyuuu/ml-service-blueprint --private --clone

  2. Install

    Virtualenv, the package, and the dev extras.

    make install

  3. Run the golden path

    Data through training, validation, artifact, version, service, Docker, CI and monitoring hooks. A stage whose prerequisites are missing is named as unverified, never quietly counted as a success.

    make golden-path

The golden path

Nine stages, one command.

Clone the repository and run make golden-path. It executes every stage below, verifies the result, and prints what passed, what failed, and what it could not check on your machine.

1.0 · DATA

A dataset that needs no network

The reference data is scikit-learn's bundled Wisconsin breast-cancer set, written to CSV by a script in the repo. Two properties make it the right infrastructure fixture: it works offline, so CI, containers and a plane-mode clone behave identically; and a full train-evaluate-register cycle finishes in under a second, so the golden path is something you actually run.

It is real data with real column names — not synthetic filler. Point the config at your own CSV and nothing downstream needs to know.

Rows
569
Features
30 numeric
Split
341 train · 114 validation · 114 test
Digest
SHA-256 recorded into every artifact

2.0 · TRAINING

Deterministic, and the schema comes from train rows only

One seed drives numpy, the estimator, and a three-way stratified split. Same config, same data file, same seed produces byte-identical scores — there is a test that asserts it, because every metric comparison between two model versions rests on that property.

The feature schema is derived from the training split alone. Deriving it from the full frame would bake test-set ranges and categories into the artifact's declared contract and its drift baseline — a subtle leak that no metric would ever reveal.

3.0 · VALIDATION

The threshold is chosen on validation. The gates are checked on test.

Three splits exist for a reason. Train fits the pipeline. Validation selects the decision threshold. Test produces the numbers you report and gate on, and influences neither.

Tuning a threshold on test data leaks the test set into the deployed decision rule and makes every reported number optimistic. That is why validation is a separate split rather than "just use cross-validation" — the threshold is a shipped artifact field, not a modelling detail.

Gates are thresholds a run must clear to be registrable. A run that misses one still writes its metrics and its model card; what it cannot do is register.

configs/training.yaml
gates:
  min_roc_auc: 0.95
  min_f1:      0.90

# a run that misses a gate:
error: evaluation gates failed:
  roc_auc=0.9312 below required 0.9500

The refusal lives in the registry, not the CLI. No path — notebook, script, Airflow task — can register a model that missed its bar without passing allow_failed_gates=True on purpose.

4.0 · ARTIFACT

One immutable bundle that can answer for itself

An artifact is a directory of three files: the fitted pipeline, a metadata document, and a generated model card. The metadata carries everything you would actually want during an incident — not a wiki page that drifted.

The model file's SHA-256 is recorded at save and re-checked at every load. Pickles execute code when they load, and a registry directory is often a mounted volume; the digest turns "someone replaced the model file" from an invisible event into a startup failure.

Pipeline
preprocessor + estimator, one object
Schema
every column, kind, range, null rate
Metrics
test + validation, plus gate failures
Provenance
git commit, dirty flag, library versions
Config
full training + model snapshot
Card
generated, so it cannot go stale

5.0 · VERSION

A registry that makes rollback a pointer move

The shipped registry is a directory. Versions are immutable and allocated with an exclusive mkdir, so two trainers racing cannot collide. Stage pointers live in a small JSON file, and every stage change is appended to a log.

That log is what makes rollback honest arithmetic rather than a guess: read the last transition for the stage, promote back to its previous version. It is implemented once on the abstract base class, so any backend that records transitions correctly inherits a correct rollback — MLflow, S3, or your own.

$ mlservice registry history
09:14  production  - → v1   ci     initial
11:02  production  v1 → v2  alice  beats v1 on recall
15:41  production  v2 → v1  bob    (rollback) score drift

6.0 · SERVICE

Two probes, because they answer two different questions

/health is liveness: is the process alive? It deliberately ignores model state. A liveness probe that fails on a missing model makes an orchestrator restart-loop a pod that a restart cannot fix.

/ready is readiness: can this replica serve? It returns 503 when no model is loaded, keeping a half-started pod out of the load balancer while leaving it running and inspectable.

Every request is validated against the artifact's frozen schema — reordered, coerced, and checked — and a bad payload comes back with every problem listed at once, not one per round trip.

POST /predict422
{
  "error": "feature_schema_violation",
  "request_id": "fe6aea24…",
  "details": [
    "missing required feature(s): mean_area, …",
    "mean_radius: 2 non-numeric value(s)"
  ]
}

7.0 · CONTAINER

Non-root, hash-locked, and honest about its size

Two stages. The builder resolves dependencies from a fully-pinned lockfile with hashes; the runtime copies only the finished virtualenv. Nothing that builds a wheel survives into the shipped image.

It runs as uid 10001 with no shell and no home directory. The healthcheck probes liveness, not readiness — the same reasoning as the probes above.

Base
python:3.11-slim-bookworm
User
uid 10001, nologin, no home
Deps
--require-hashes, runtime only
Size
530 MB — measured, not estimated

530 MB is not small, and saying otherwise would be a lie. scipy, pandas, scikit-learn and numpy are about 190 MB unpacked between them. The only lever with real leverage is exporting to ONNX — at the cost of the training/serving parity this design exists to protect. Trimming layers will not get you there.

8.0 · CI

Six jobs, separate on purpose

A container build failure should not hide a lint failure, and the golden path should be readable as its own result. Lint and strict typing run on both 3.11 and 3.12; the golden path runs end to end; the package builds and passes twine check; the container is built and smoke-tested against a live container; dependencies are audited against the lockfile and the lockfile itself is checked for drift.

CI is not decoration here. It caught five defects in this repository's own first commit that every local check had passed — including a .gitignore rule that silently kept an entire source package out of the initial commit.

9.0 · MONITORING

Extension points, not a drift product

Two seams and a refusal to grow past them. DriftSink is where scored predictions go — write one class for Kafka, BigQuery, or a vendor SDK. DriftDetector turns records into signals, offline, against what the sink wrote.

Between them sits a reporter that enforces two properties the route handler must not be trusted to remember.

Fail-open
a sink that raises leaves /predict at 200 — verified by test
Sampled
at volume, the emit is the expensive part
PII
feature values withheld by default
Signal
score-distribution PSI, no labels needed

The one idea

The artifact owns everything.

The fitted estimator, the fitted preprocessor, the feature schema, the decision threshold, the metrics it was accepted on, and the code that produced it all live inside a single immutable artifact. The service reimplements none of it.

That closes off training/serving skew structurally, rather than by discipline. A visible consequence: scoring a CSV from the CLI and calling POST /predict run the same code path. They cannot disagree.

Usual cause of skewWhy it can't happen here
Preprocessing rewritten in the serviceThe preprocessor is inside the serialized pipeline. There is no serving-side feature code to diverge.
Columns in a different order, or the wrong dtypeEvery request is reordered and coerced against the schema frozen at training time.
Threshold drifting from the validated oneThe threshold is a field of the artifact, chosen on validation, served from metadata.

Verification status

What is checked, and where.

This table distinguishes what runs on a developer's machine from what only a CI runner can prove. Docker is not installed on the machine this was authored on, so the container is verified in CI and reported locally as skipped — never as passing.

AreaVerifiedNotes
Training, determinism, gatesLocally + CISame seed reproduces byte-identical scores
Artifact contract, digest, round-tripLocally + CIA tampered model file is rejected at load
Registry promote / rollback / historyLocally + CIRollback restores the previous version
Service, schema enforcement, batchingLocally + CIAlso exercised against a live uvicorn server
Logs, metrics, drift fail-openLocally + CIA raising sink leaves inference at 200
Container: builds, boots, serves, non-rootCI onlyNo Docker daemon locally; reported as SKIP there
Throughput and latencyNot measuredk6 and Locust profiles ship; every threshold is marked PLACEHOLDER

No throughput numbers appear anywhere in the repository, deliberately. Latency for an in-process scikit-learn model depends on your estimator, feature count, batch size, CPU allocation and co-tenants. A number measured on a laptop would read as a specification and would be wrong for you.

Replacing the example

Your model, the same infrastructure.

Point the config at your CSV, name your target column, list the identifiers and leaky fields to drop, choose an estimator, then set the gates from what you actually measured. You do not touch the service, the registry, the observability layer, the Dockerfile, or CI — they read the feature set from the artifact instead of hard-coding it.

Use this template Architecture notes