Case study 01
Commercial credit-risk modeling
Build the outcome labels first. Then earn a score.
Tested whether commercial credit signals separate businesses with observed adverse outcomes, then built a regularized ranking score that now ships in a business-credit product.
- Role
- Data Science Co-op
- Period
- 2026
- Status
- Shipped score · separate research
01 — My contribution
I built the labelled outcome dataset, screened the available signals, developed and evaluated the model, and packaged the score with reference examples so the product implementation could be tested against it.
02 — Problem and context
A business-credit report shows several scores, but there was no in-house evidence that they separate businesses that later run into trouble from businesses that do not. Without outcome labels, choosing between signals was a matter of opinion.
The work had two questions. Do the existing signals rank businesses with observed adverse outcomes below the general population? And can a small, reviewable model rank them better than any single signal?
From public filings to a tested score
Public filings
Filter to business entities
Normalize + link
Staged name matching; confident links only
Enrich
Attach business records
Screen signals
Direction, discrimination, leakage
Regularized model
Percentile-binned inputs
Reference examples
Python ↔ TypeScript parity
03 — Approach and key decisions
Decision 01
Build labels from public records
No labelled outcome set existed, so I built one. Public business filings were filtered to business entities, names were normalized, and each filing was linked to a commercial business record through staged matching: exact first, then known aliases, then guarded fuzzy matching. Only confident links entered the labelled set; ambiguous ones were left out rather than guessed.
Decision 02
Compare against the population, not a convenience sample
Comparison records came from a seeded random sample of the wider business population. The model therefore learns to separate outcomes from the kind of business a customer would actually look up.
Decision 03
Screen every signal before modeling
Each signal was evaluated on its own: does it discriminate at all, does it point the expected direction, and could its values be contaminated by information recorded after the outcome? Signals that failed were excluded before any model was fit.
Decision 04
Prefer a small, transparent model
I used regularized logistic regression on percentile-binned inputs. Binning handles heavy tails and heaped values without assuming a functional form, regularization keeps the bin weights stable, and the whole model can be reviewed line by line.
Decision 05
Make the production port provable
The model was developed in Python; the product is written in TypeScript. I packaged a dependency-free reference scorer and a set of reference examples, so the product implementation is tested against exact expected outputs instead of being trusted by inspection.
04 — Evaluation and evidence
The goal was to report how the score behaves under several realistic conditions, not one flattering number.
- Untouched held-out test set
- Set aside before any modeling and used for the final comparison only.
- Temporal and backward checks
- Trained on one period and tested on another — both later and earlier outcomes that never touched training — to see whether the ranking survives a change of period.
- Prospective forward check
- Tested on outcomes that occurred after the data snapshot was taken, the closest analogue to real use.
- Baseline comparison
- Compared with the existing signals on the same records and the same protocol.
- Leakage and selection-bias review
- Checked whether missing-value handling, or the matching process itself, was manufacturing signal. Larger businesses are easier to match, for example, which can make size look predictive for the wrong reason. The frame was corrected where it was.
- Reproducible scoring
- The Python reference and the TypeScript implementation agree on every reference example.
05 — What shipped
The score is integrated into a business-credit product as a customer-facing ranking.
For the overall verdict shown to customers, the product needed something a reader could reconcile with the individual scores on the page. The shipped verdict is a transparent rule chosen by systematic search over candidate rules, not one of the research models.
- Status: Shipped
Ranking score
Customer-facing ranking score with Python/TypeScript parity tests.
- Status: Research
Additional model families
Explored more flexible models to understand the performance ceiling and test assumptions. Not shipped.
- Status: Shipped · rule-based
Overall verdict
A transparent rule designed so the verdict never contradicts the scores beside it.
06 — Limitations and lessons
Limitations
- Labels depend on matching. Outcomes that could not be linked confidently are missing, and matching is easier for some kinds of businesses than others.
- How comparison records are sampled shapes what the evaluation measures.
- This is a ranking tool. It is not a calibrated probability of default, and it is not deep learning.
- A forward period was checked, but performance needs ongoing monitoring as data and conditions change.
Lessons
- Most of the difficulty was in constructing trustworthy labels, not in fitting the model.
- Several validation views reported together are more honest than a single best result.
- Leakage checks earned their keep: one apparent improvement came from how missing data was handled, not from risk.
07 — Tools
- Python
- DuckDB
- NumPy
- pandas
- SciPy
- scikit-learn
- Jupyter
- TypeScript