datanalysis-credit-risk

The datanalysis-credit-risk skill is a structured data preprocessing and variable screening pipeline designed specifically for credit risk modeling in pre-loan scenarios. It addresses common challenges in raw credit datasets such as inconsistent formatting, missing values, noisy features, unstable variables across time, and redundant or highly correlated predictors. By standardizing and systematizing the data cleaning process, it helps ensure that downstream credit risk models are built on high-quality, reliable, and statistically meaningful features.

The skill implements an end-to-end 11-step pipeline that operates in a non-destructive manner, meaning the original dataset is preserved throughout the process. It begins with data loading and formatting, followed by organizational-level sample analysis and separation of out-of-sample (OOS) data. It then performs abnormal month filtering based on sample size constraints, missing value analysis, and progressive feature elimination. Features with excessive missing rates are removed first, followed by low information value (IV) variables, unstable features based on population stability index (PSI), noisy features identified through null importance permutation, and highly correlated variables. Finally, it generates a comprehensive cleaning report summarizing all transformations and decisions applied during the pipeline.

This skill is primarily intended for credit risk analysts, data scientists, and machine learning engineers working on pre-loan underwriting systems or financial risk modeling pipelines. It is especially useful when dealing with raw banking or lending datasets that require rigorous feature screening before model training. It ensures improved model stability, interpretability, and predictive performance while maintaining traceability through automated reporting.