Drift Donut

What Drift Donut Actually Does

Drift Donut is a free, open-source tool for monitoring data drift in machine learning models. It sits between your training data and your production data, comparing distributions to flag when things have shifted enough to worry about. The visual output is a donut-shaped chart that shows drift percentage across features, which is where the name comes from. It's built on Python, uses statistical tests under the hood, and is designed to integrate into CI/CD pipelines without requiring a PhD in statistics. I've been running drift monitoring on production models since before these kinds of tools became mainstream, and I still reach for Drift Donut when I need something fast to get up and running. It handles numerical and categorical features out of the box. It doesn't require a database or a cloud subscription. You point it at two datasets, it runs, you get a result. That's about it.

How to Set Up Drift Donut

The installation is straightforward. You'll need Python 3.8 or higher installed. Run pip install drift-donut. That's the main package. There are a few optional dependencies you might want depending on your setup, but the core functionality works with just the base install. I'd recommend also installing pandas and numpy if you aren't already using them. Once it's installed, you load your reference dataset — that's your training data, the baseline you're comparing against — and your target dataset, which is whatever new production data you're checking. Here's what the basic call looks like: reference_data = pd.read_csv("training_data.csv")
production_data = pd.read_csv("production_data.csv")

from drift_donut import DriftDonut
dd = DriftDonut(reference_data)
result = dd.analyze(production_data) The analyze method returns a dictionary with drift scores per feature and a summary. You can also generate the donut chart visualization by calling result.plot(). I usually save the output to a JSON file so I can track drift over time rather than just looking at a single snapshot. One thing I wish was better documented is how to handle missing values. Drift Donut will flag features with high missingness in the production set as drifted, but the default behavior for imputation isn't always what you want. In my experience, the safest approach is to clean your data before running the analysis rather than relying on the tool to handle it. I ran into an issue where a feature had 40 percent missing values in production but only 2 percent in training, and the drift score was misleading because the algorithm was trying to compare distributions with fundamentally different completeness levels. My workaround was to add a simple preprocessing step that filters out any feature missing more than 10 percent of values before passing it to Drift Donut. This usually cuts false positives down significantly.

Reading the Results

The donut chart itself is useful for quick reporting, but the real value is in the underlying numbers. Each feature gets a drift score based on a statistical test — typically the Kolmogorov-Smirnov test for numerical features and the chi-squared test for categorical ones. The threshold for what counts as "drifted" is configurable, and the default of 0.05 p-value is reasonable for most use cases but not universal. Here's the counter-intuitive part that most beginners miss: a statistically significant drift result doesn't necessarily mean your model needs retraining. Drift Donut will flag tiny distributional shifts that are statistically significant but practically irrelevant, especially with large sample sizes. I've seen production systems trigger drift alerts on features with changes so small they had zero impact on model performance. The rule of thumb I use is that you should look at both the p-value and the effect size. If the drift is statistically significant but the effect size is negligible, you can usually ignore it. Drift Donut provides effect size metrics, but they're buried in the output dictionary rather than highlighted in the visualization. Another thing people don't realize is that Drift Donut compares feature distributions independently. It doesn't account for correlations between features shifting together. If your model relies on the relationship between two features rather than their individual distributions, you might get clean per-feature results while the model degrades silently. I encountered this on a credit scoring model where income and debt_ratio shifted in a coordinated way that preserved their correlation but broke the model's decision boundary. The workaround was to create interaction features before running the drift analysis, or to supplement Drift Donut with a multivariate approach like PCA-based monitoring for the critical cases.

Drift Donut in a Pipeline

Integration into an existing pipeline typically means running the analysis on a schedule or after each batch of new data. I set up a cron job that pulls the latest production data, runs Drift Donut against the training baseline, and posts the results to a Slack channel. If any feature exceeds the drift threshold, it triggers a page. This has been running for about eight months on three different models without major issues. For batch processing, you can pass multiple production datasets at once and get cumulative drift tracking. The tool stores a history of comparisons if you configure the output path correctly. I keep a weekly JSON file for each model, and I've found that reviewing the drift trajectory over time is more informative than any single measurement. A feature that drifts gradually over six weeks is often less concerning than one that spikes suddenly and then stabilizes, even if the cumulative drift is higher in the first case. The main limitation I run into regularly is that Drift Donut doesn't handle time-series data well without modification. If your production data has temporal ordering that matters, the random sampling approach it uses by default can mask important patterns. I solved this by wrapping the analysis in a function that splits data by time windows and runs Drift Donut on each window separately, then aggregates the results. It's not built in, but it's a few lines of code.

If you're working with very high-dimensional data — thousands of features or more — Drift Donut will slow down considerably. The statistical tests don't scale linearly, and you'll see execution times jump from seconds to minutes. For those cases, I recommend filtering features down to the most important ones first, either through model feature importance or correlation analysis, and then running Drift Donut on that subset. This usually gets you the signal you need in under a minute instead of fifteen. There are alternatives if Drift Donut doesn't fit your needs. Evidently AI is more feature-rich but requires a heavier setup. Alibi Detect is good for deep learning workflows. But for a lightweight, no-frills drift check that you can drop into an existing project without architecting a monitoring platform, Drift Donut is hard to beat. It does one thing and does it reasonably well. The GitHub repository is at drift-donut/drift-donut and the documentation covers the API reference in more detail. I'd suggest cloning the repo and running the example notebooks if you're new to it. The code is clean and the issues section has answers to most of the edge cases you'll run into.