Churn model the call centre acts on every morning

The call centre needed to know whom to call first to keep a player before they leave. I built a model that every day estimates the probability of leaving within 30 days from activity, trends, payments and negative experiences. On the same players it is more precise than the ML team's model: in the top 20k of the list, 55.7% actually leave versus 46.3%. After the model once degraded, I added a check before every retrain and a drift monitor. It now also runs for a second business line, and the effect of the calls themselves is measured in a separate A/B test.

Input

42 features: sessions, 7- and 30-day trends, negative experiences, payments.

Model

CatBoost: ROC-AUC 0.756 vs 0.675 for the ML team's model on the same 29.6k players.

Action

Ranks who will leave in the next 30 days, so the call centre calls them first.

Guards

Training gate and drift monitor added after a degradation episode; calibration within ±4 pp; carried over to a second business line.

Impact

Precision on the top 20k: 55.7% vs 46.3%; ROC-AUC 0.85 in production.