Drift-aware retraining with shadow-to-active promotion gates

Detecting drift before it costs you

MLA monitors two signals over a rolling sample window: F1 for degrading predictive performance, and PSI (Population Stability Index) for feature-distribution shift. When either crosses its configured threshold, it triggers a SMOTE-balanced XGBoost retraining run so the model relearns against the current fraud landscape rather than last quarter’s.

Crucially, it learns from groundTruthFraud overrides — chargebacks, disputes, and reviewer decisions fed back through the Labels API — not from the model’s own past predictions. Training on your own outputs compounds error; training on ground truth corrects it.

Nothing reaches production unproven

Every candidate model moves through explicit promotion gates: CANDIDATE → SHADOW → ACTIVE. Promotion to ACTIVE requires a McNemar significance check (p < 0.05) and a real improvement of ΔF1 ≥ 0.01 — a new model has to be statistically better, not just newer.

A replay CLI lets you re-score any candidate against the live audit log without touching production traffic, so you can see exactly how a model would have decided real historical transactions before it ever goes live.

• Monitors F1 (poor performance) and PSI (feature distribution shift) over a rolling sample window.
• Triggers SMOTE-balanced XGBoost retraining when either signal crosses its configured threshold.
• Promotion gates: CANDIDATE → SHADOW → ACTIVE, each requiring explicit approval or auto-promote rules.
• McNemar significance check (p < 0.05) and ΔF1 ≥ 0.01 required before any promotion to ACTIVE.
• Replay CLI lets candidate models be re-scored against the live audit log without going to production.
• Learns from groundTruthFraud overrides, not from the model’s own past decisions.