How ChurnLens predicts churn — honestly
ChurnLens is a productized, leakage-correctedversion of our peer-reviewed research on telecom churn. Below is the whole method in plain language — including exactly where the original paper's methodology was too optimistic, and what we changed.
The correction, up front
The paper applied SMOTE to the whole dataset before splitting. That lets synthetic copies of a churner land in both train and test — the model is quietly graded on data it has already seen, and the headline 94% accuracy is inflated. We moved SMOTE inside every cross-validation fold, so it only ever touches training rows. The honest number is a cross-validated ROC-AUC of 0.911 ± 0.016 — and because churn is only 14% of customers, accuracy was never the right yardstick anyway.
Based on published research
Predicting Customer Churn in Telecom Industry: A Machine Learning Approach for Improving Customer Retention
A. Patel and A. G. Kumar, 2023 IEEE 11th Region 10 Humanitarian Technology Conference (R10-HTC), 2023.
Read & understand your columns
ChurnLens classifies every column — a number (monthly spend), a yes/no flag (has international plan), a category (plan tier), or an ID — auto-detects the churn label, and asks you to confirm it before anything runs.
Balance classes — but only inside each fold
Churn is rare (14% of customers here). SMOTE (Synthetic Minority Over-sampling) fixes the imbalance by interpolating new churner examples from real ones. The catch: do it once on the whole dataset and you leak — a synthetic point built from a test-set churner ends up training the model.
So we run it as a step insidean imbalanced-learn pipeline, refit separately on each cross-validation fold's training rows. No test row is ever synthesised from. This is the single change that turns the paper's optimistic numbers into honest ones.
Cross-validate — report mean ± std, not one lucky split
A single train/test split is a coin flip. We use 5-fold cross-validation and report the spread, so you see how stable the model really is.
| Model | CV ROC-AUC (mean ± std) |
|---|---|
| Logistic Regression | 0.800 ± 0.027 |
| Random Forest | 0.910 ± 0.011 |
| Gradient Boosting← shipped | 0.909 ± 0.016 |
Gradient boosting (the XGBoost family that topped the paper's benchmark) wins; tuned, it reaches 0.911 CV ROC-AUC and 0.913 on a fully held-out test set.
Calibrate the probabilities
SMOTE balances the training data to ~50/50, so the raw model talks as if churn were a coin flip. We fit two calibrators — Platt (a sigmoid) and isotonic (a monotonic step fit) — and keep whichever scores the lower Brier score, mapping scores back to the true 14% base rate.
Here isotonic won (Brier 0.0463 vs 0.0465). The result: when ChurnLens says “30% risk,” about 30% of those customers really do churn — so the number, and the revenue-at-risk it implies, can be trusted.
Explain every prediction with SHAP
For each customer we compute exact TreeSHAP values — the same algorithm as the Python shap package — showing precisely which features pushed theirrisk up or down, and by how much (on the log-odds scale the model reasons on). No heuristics: the bars are the model's actual arithmetic.
Across the dataset, the strongest drivers are Customer service calls, Total day minutes, International plan — the international-plan and frequent-support-call signals the paper called out, now quantified per customer.
Pick your decision threshold
A calibrated probability isn't a yes/no — you choose the cut-off. The threshold slider shows precision, recall, and F1 at every value (measured on held-out data) so you can decide whether to catch more churners (higher recall) or flag only the most certain (higher precision). The at-risk list and revenue-at-risk update live.
Turn it into an action plan
Finally we group at-risk customers by their top shared factor into cohorts and emit one prioritized recommendation per cohort — largest, most valuable first.
Same model, two runtimes — verified identical
The model is trained once in Python (scikit-learn + imbalanced-learn), then exported to JSON. It runs two ways: a tiny FastAPI endpoint serving the scikit-learn model with exact SHAP, and a native TypeScriptport that runs in your browser. A build-time test asserts both reproduce scikit-learn's probabilities — and the SHAP values reproduce the shap package — to within 10⁻⁶. No black box, no drift.
A note on privacy
When you upload your own CSV, the entire pipeline above — cross-validation, SMOTE, calibration, SHAP — runs inside your browser in a background thread. Your customer data is never uploaded. If you create an account, only the aggregate results are saved, never the raw rows.