Finding the future VIPs the main model misses

The main ML model found 41% of future VIP players; the rest went unnoticed. I built a model that looks at payment behaviour in the first two and three weeks after the first payment and searches for future VIPs among players the main model did not flag. Validation is time-based so the model never peeks into the future, and the first production run passed all 295 checks. VIP managers now get 27% more leads every day with no overlap with the main model; players in the top 1% of the forecast become VIPs 16 times more often than average.

Input

Payment behaviour on day 14 and day 21 after the first payment, only for players the ML model did not flag.

Model

CatBoost, ROC-AUC 0.87–0.89. Players in the top 1% of the forecast become VIPs 16 times more often than average.

Action

VIP managers get 27% more leads every day, none of them overlapping the main model.

Guards

Time-based split; the first production run passed 295 of 295 selection checks.

Impact

The day-21 checkpoint added another +40% leads at the same precision.