Credit Risk Model Stability

A research project exploring credit risk prediction using ensemble methods on Home Credit's consumer lending data. The focus is on building stable, high-performing default prediction models through gradient boosting ensembles and robust feature engineering.

Overview

This project tackles the problem of predicting whether a borrower will default on a loan. We approach it as a binary classification task, combining multiple gradient boosting frameworks into a weighted voting ensemble to maximize predictive performance and stability over time.

Methodology

Data Processing

• Data is loaded from multi-depth parquet files covering applicant demographics, credit bureau records, tax registries, previous applications, and more
• Feature engineering aggregates depth-1 and depth-2 relational tables into case-level features using statistical summaries (max, last, mean, variance, count)
• Date features are transformed into relative durations from the decision date
• Columns with >70% missing values or low/excessive cardinality are filtered out
• Correlated features (threshold 0.8) are grouped and deduplicated to reduce redundancy
• Memory optimization downcasts numeric types to reduce footprint

Models

Three gradient boosting classifiers are trained independently using Stratified Group K-Fold cross-validation (5 folds, grouped by week to respect temporal ordering):

Model	Framework	Best CV AUC
CatBoost	catboost	0.774
LightGBM	lightgbm	0.789
XGBoost	xgboost	0.777

Ensemble

A custom VotingModel combines predictions from all 15 trained models (5 folds x 3 frameworks). LightGBM predictions are double-weighted in the average, reflecting its stronger individual performance.

Project Structure

├── Home Credit LGB Cat Ensemble.ipynb   # Main notebook (data pipeline, training, inference)
├── Paper Credit risk model stability.pdf # Research paper
└── README.md

Requirements

• Python 3.8+
• numpy, pandas, polars
• scikit-learn, imbalanced-learn
• lightgbm, xgboost, catboost
• seaborn, matplotlib

Dataset

This project uses the dataset from the Home Credit - Credit Risk Model Stability Kaggle competition.