Building a Data Science Workbook That Doesn't Fall Apart After Three Uses

I built my first proper monthly data science workbook about eighteen months ago and immediately hit the wall that everyone hits. The template looked clean in the morning and by Thursday I was drowning in scattered Jupyter notebooks, three different conda environments, and a results folder that contained no traceable link back to the code that produced it. Here is how I got out of it and what actually stuck.

Monthly Data Science Workbook: What It Actually Is

It is not a product you download from a store. A Monthly Data Science Workbook is a repeatable, self-contained project scaffold that you reset or branch at the start of each calendar month and use to house explorations, experiments, and production-relevant analysis without letting them corrupt each other. The "monthly" part is deliberate. It forces you to commit, archive, and close a loop every thirty days instead of letting your work become an eternal scratchpad that nobody can reproduce six months later. I learned this the hard way after a client asked me to re-rerun a model we had apparently "finalized" in February. My February notebook was still open in VS Code, my environment variables had drifted across four separate virtualenvs, and the training data I thought I was using was actually a stale CSV I had renamed but never deleted. It took me six hours to reconstruct what should have taken twenty minutes. That is the exact pain a workbook structure is supposed to prevent.

The Structure I Actually Use

Here is the directory layout. It is boring on purpose. ``` workbook-2024-01/ data/ raw/ processed/ external/ notebooks/ 00-explore.ipynb 01-preprocess.ipynb 02-feature-engineering.ipynb 03-training.ipynb 04-evaluate.ipynb 05-experiment-log.ipynb src/ config.py data_pipeline.py features.py model.py evaluate.py scripts/ fetch_data.sh train.sh evaluate.sh outputs/ models/ figures/ reports/ configs/ default.yaml ablation-v1.yaml requirements.txt pyproject.toml README.md .gitignore .env.example ``` The numbering in the notebooks is not aesthetic. It enforces execution order so you do not accidentally run `04-evaluate` before `03-training` and then spend an hour wondering why your ROC curve is empty. I used to skip this and regret it every single time. The `src/` folder exists because importing from notebooks is a well-documented anti-pattern. When I put training logic inside a notebook cell and then tried to reuse it in a script three weeks later, Python imported a stale version of the module from `__pycache__` and my eval numbers diverged from what I thought I had logged. Moving everything into `src/` and importing it with a proper package structure fixed that. I switched after burning two days on a reproduction bug that turned out to be a cache issue, not a model issue.

How I Set It Up in Practice

I start each month with a fresh branch or a completely new directory if the previous month's work is archived. The command I run is basically: ```bash cp -r ../workbook-template/ workbook-$(date +%Y-%m) cd workbook-$(date +%Y-%m) uv venv uv pip install -r requirements.txt cp .env.example .env code . ``` I use `uv` now instead of `conda` because environment creation was eating too much of my morning. Conda is fine for CUDA-heavy GPU work, but for most tabular and Transformer-based workloads `uv` gets you a clean environment in about eight seconds instead of forty-five. I kept wasting time on solver conflicts during the first week of each month. This removed that friction entirely. For configuration, I use a single YAML file per experiment run. The `default.yaml` contains dataset paths, random seed, model architecture flags, and output directories. Each ablation or parameter sweep gets its own YAML file that overrides only what changes. I used to pass arguments via command line and then lose track of which seed produced which result. The YAML approach is slower to type but cuts my post-hoc debugging time by roughly sixty percent because the config is visible in git history. The `scripts/train.sh` file is the entry point. It looks like this: ```bash #!/usr/bin/env bash set -euo pipefail CONFIG="${1:-configs/default.yaml}" export PYTHONPATH="$(pwd)/src:${PYTHONPATH}" python -m src.model train --config "$CONFIG" ``` Simple, but the `set -euo pipefail` part matters. Without it, a failed data fetch silently produces an empty dataset and your model trains on nothing while you celebrate false accuracy numbers. I saw this happen once during a production migration and it took three days to notice because the validation loss looked deceptively flat.

The Parts That Actually Break

A workbook like this fails in three predictable ways. First, the data versioning problem. If you do not pin your raw data to a specific snapshot or hash, your results become unreproducible the moment someone updates the source CSV. I solved this by storing a `.sha256` file next to each dataset and checking it in the `fetch_data.sh` script. The check takes two seconds and prevents the most common source of "why did my numbers change" panic. Second, notebook bloat. A single exploratory notebook will grow to four hundred cells if you let it. I now enforce a hard rule: if a notebook exceeds fifty cells, I extract the recurring logic into `src/` and replace the cells with imports. It feels inconvenient at first because you have to write the module first, but it saves hours of triage later when you need to find which three lines produced a specific confusion matrix. Third, environment drift. Even with `uv`, dependencies update. I pin the major versions in `requirements.txt` and run `uv pip freeze > requirements.txt` at the end of each month. The resulting file becomes the baseline for the next month's copy. This means my January workbook and February workbook share an identical dependency tree unless I intentionally change something.

How to Download or Replicate This

There is no single installer for a Monthly Data Science Workbook because the value is in the structure, not in any particular script. You can create one in about ten minutes by copying the directory layout above and filling in the files as you go. If you want a starter template, the most practical approach is to clone a minimal repo and rename it each month. I keep mine on GitHub as a private repository with a `templates/workbook-base/` folder. When I start a new month, I run: ```bash git clone https://github.com/your-username/workbook-base.git workbook-2024-02 ``` Then I customize the config files and start working. This takes about three minutes end-to-end and gives you a clean slate with zero risk of mixing last month's artifacts into this month's analysis.

Counter-Intuitive Things Beginners Miss

Most people focus on the notebooks and ignore the `outputs/` directory structure. They put models, figures, and reports in the same folder and then cannot tell which model checkpoint corresponds to which experiment. I learned to separate them strictly: `outputs/models/` for serialized weights, `outputs/figures/` for PNGs and SVGs, `outputs/reports/` for markdown summaries. A model and a figure are fundamentally different artifacts and treating them the same creates unnecessary cognitive load when you are trying to debug something at 11 PM. Another thing nobody warns you about: the random seed is not enough. If you are using data loaders with multiple workers, shuffling, and non-deterministic GPU operations, setting `seed=42` will not make your results reproducible across machines. I discovered this when I moved a model from my laptop to a cloud GPU and got different training curves despite identical code and configs. The workaround is to pin the PyTorch deterministic flags, disable data loader shuffling during training, and log the full environment state including CUDA version and driver. It adds about thirty seconds to each run but eliminates the "it worked on my machine" class of bugs entirely.

When This Approach Fails

A structured monthly workbook is overkill for one-off analyses that you will never revisit. If you are doing a single exploration for a Slack message and the answer will be irrelevant in forty-eight hours, the overhead of setting up `src/`, configs, and scripts is genuinely wasteful. In those cases, a plain notebook in a temporary folder is faster and morally equivalent. It also struggles with highly interactive, iterative work where you are constantly jumping between data loading, visualization, and model tuning in the same session. The strict notebook ordering that works for pipeline-style projects becomes friction when you are doing rapid prototyping. I keep a separate `scratch/` folder for that kind of work and only promote code to the main workbook when it passes a basic reproducibility check. For team settings, the workbook model assumes a single owner or a very small group. Once you have five people pushing to the same `notebooks/` folder, merge conflicts in Jupyter files become a full-time job. In that scenario, I recommend moving to a DVC-based pipeline or a notebook governance tool instead of trying to force a personal workbook structure onto a team workflow.

What I Check Before Closing Each Month

I run through a quick checklist on the last day: - All raw data has a `.sha256` file and the checksum matches - No notebook exceeds fifty cells - `requirements.txt` reflects the current pinned dependency state - The `outputs/` directory contains no unlinked artifacts - The README has been updated with what changed this month - The previous month's workbook is archived or pushed to the remote This takes about twelve minutes. Skipping it usually means I spend six hours reconstructing context the following month. I stopped skipping it after the February incident I mentioned earlier and the cumulative time savings have been substantial. The workbook itself is a disciplined container, not a magic solution. It will not fix bad feature engineering, unrealistic evaluation metrics, or unclear project scope. But it does prevent the most common form of technical debt in data science work, which is the slow accumulation of invisible assumptions that only surface when someone asks you to reproduce a result you already consider settled.