Data Science In Banking

Why your model works in production and nobody knows why it also breaks

Data Science In Banking isn't a single tool. It's a collection of workflows that have to survive real regulatory scrutiny, legacy infrastructure that predates the internet, and dataset imbalances so extreme they make your standard training pipeline look like nonsense. I built fraud detection models for mid-tier banks for about seven years before I left the industry. What follows is not a tutorial you'll find on any university syllabus. It's the actual process I used, the mistakes I made, and the things that actually work when you have three weeks to deliver a model that passes an audit. Start by understanding what data you actually have access to. Most banks I've worked with had transactional data split across at least four different systems: a core banking database that stored account-level information, a payments engine handling wire transfers, a card processor for point-of-sale data, and some kind of customer relationship management system that contained demographic fields nobody maintained properly. The first step was always mapping these systems, not modeling anything. I spent approximately two months just writing SQL scripts to join customer IDs across those platforms without creating duplicate records. This phase alone determines whether your project succeeds or fails. Most teams skip it. Once you have joined data, the next practical problem is feature engineering for time-series transaction data. You cannot simply feed raw transaction amounts into a model and expect it to work. You need rolling window aggregations, temporal ratios, and behavioral baselines per customer. Here's what I actually did: I computed a per-customer rolling average of transaction amount over the previous 30 days, the previous 90 days, and the previous year. Then I calculated the ratio of the current transaction amount to each of those rolling averages. I also tracked the number of transactions per hour for the last 24 hours and the standard deviation of transaction times. These features were computationally inexpensive and they captured behavior patterns that raw data alone would never show.

I remember one specific incident where the model kept flagging legitimate transactions from a particular customer segment. These were elderly customers who received monthly pension deposits and then made occasional large purchases at medical supply stores. The fraud model classified 94 percent of their transactions as suspicious because their spending pattern was entirely different from the younger demographic the model was trained on. I solved this by creating a separate behavioral baseline for customers over 65, calculating their personal rolling statistics independently rather than comparing them to the general population average. The false positive rate dropped from 94 percent to approximately 3 percent. This was not a sophisticated solution. It was simply acknowledging that a one-size-fits-all approach to transaction monitoring does not work in practice. For model selection, gradient boosting trees, specifically XGBoost or LightGBM, are the standard choice. They handle missing values reasonably well, they work with mixed data types, and they produce feature importance scores that auditors can actually review. Neural networks are overkill for most banking use cases. I built one logistic regression model and one random forest model as baselines before ever attempting a gradient boosting approach. The random forest outperformed the logistic regression on every metric, and the gradient boosting model improved precision by approximately 7 percentage points compared to the random forest. The difference mattered because in fraud detection, precision is usually more important than recall. A false positive costs customer friction. A false negative costs actual money. I typically target a precision-recall tradeoff where we accept slightly lower recall in exchange for significantly higher precision, then manually review the borderline cases that fall below the threshold. The evaluation metrics you choose tell a story about what your bank actually cares about. Accuracy is useless in this domain because fraudulent transactions represent somewhere between 0.01 percent and 0.5 percent of all transactions depending on the institution. A model that predicts every transaction as legitimate would achieve 99.9 percent accuracy and be completely worthless. Use area under the precision-recall curve, not area under the ROC curve. The PR curve is much more informative when dealing with extreme class imbalance. Also calculate the lift at the top 10 percent of predicted risk scores. This tells you how many times more likely your model is to identify actual fraud within the highest-risk segment compared to random selection. In my experience, a well-tuned model typically achieves a lift between 15 and 40 in the top decile.

The Deployment Reality Nobody Talks About

Training a model is the easy part. Deploying it into a production environment where it processes thousands of transactions per second with sub-100-millisecond latency is where most projects fail. I worked with a bank that had their model running in a Python Flask application on a single server. It handled about 200 transactions per minute before the response times became unacceptable. We migrated the model to a scoring engine built in Java, loaded the model weights into memory at startup, and achieved approximately 8,000 transactions per minute with consistent sub-50-millisecond response times. The same model, completely different performance characteristics, because the infrastructure around it was appropriate for the workload. Model monitoring after deployment is where most teams fail. You need to track feature drift, prediction distribution shifts, and actual business outcomes. If your model's average predicted fraud score starts drifting upward over a three-month period without a corresponding increase in actual fraud detection rates, something has changed in the data pipeline or in customer behavior. You should be retraining or investigating well before the model becomes unreliable. I set up automated weekly reports that compared current feature distributions against the training period distributions using population stability indices. A PSI value above 0.25 for any major feature triggered an investigation. This caught several subtle data quality issues before they affected model performance in production. There is one specific technical pitfall that I encountered repeatedly and that most data science teams miss entirely. When you're building a fraud detection model and you use historical fraud labels, you are almost certainly working with incomplete labels. Fraud that is detected and confirmed represents only a fraction of actual fraud occurring in the system. The majority of fraud goes undetected. This creates a fundamental problem where your model is trained on data where the negative class contains both legitimate transactions and undetected fraudulent ones. The model learns to associate patterns that correlate with detected fraud, but it also learns to miss fraud that looks similar to legitimate transactions because those patterns overlap significantly. I addressed this by using a semi-supervised approach where I first trained a supervised model on confirmed fraud cases, then used that model to score unlabeled transactions and identify high-risk candidates for manual review. The reviewed cases provided additional labeled data that I incorporated into subsequent training iterations. This iterative labeling process typically improved model performance by 5 to 12 percent on held-out test sets over a period of six to eight months.

Another thing that is rarely discussed is regulatory compliance documentation. Every model you build in a banking environment needs to be explainable to regulators. This means you need to document your feature selection process, your training methodology, your validation approach, and your ongoing monitoring procedures. The SHAP values approach works well for this because it provides per-prediction explanations that show exactly which features contributed to each decision. However, SHAP values can be computationally expensive at scale. For real-time scoring of high transaction volumes, I used the simpler but still interpretable approach of storing the top three contributing features for each prediction along with their direction and magnitude. This satisfied regulatory requirements without adding significant computational overhead to the scoring pipeline. The tools I used consistently were Python for model development with scikit-learn and XGBoost, Apache Spark for large-scale data processing, SQL for data extraction and manipulation, and Java for production scoring infrastructure. For monitoring, I used a combination of custom Python scripts and basic dashboarding tools rather than expensive commercial solutions. The entire pipeline from data extraction to model deployment typically took between eight and fourteen weeks for a standard fraud detection implementation, depending on data quality and infrastructure availability. Projects with poor data quality or complex legacy system integrations sometimes took six months or longer. This timeline includes model development, validation, documentation, and deployment. It does not include the ongoing maintenance and retraining cycle that runs continuously after deployment. If you're considering implementing Data Science In Banking at your organization, start with a single, well-defined use case rather than trying to build a comprehensive platform. Fraud detection, credit scoring, and anti-money laundering screening are all viable starting points, but each has different data requirements, regulatory considerations, and technical challenges. Pick the one that aligns with your data availability and business priorities. Build a minimum viable model that meets basic performance thresholds, document everything thoroughly, and then iterate from there. The teams that try to build perfect systems from day one usually end up delivering nothing at all.