Data Analysis Plan Example

Writing a Data Analysis Plan That Doesn't Waste Your Time

Data Analysis Plan Example for a Production Launch

A data analysis plan is the document you write before you touch the data. It tells you what you're trying to answer, where the data lives, how you'll clean it, and what tests or models you'll run. Without one, you spend three weeks collecting data and another two figuring out what question you were actually trying to answer. I learned this the hard way on a client project in 2021. We were measuring the impact of a checkout flow redesign. I skipped the plan, jumped straight into SQL, pulled every metric we could think of, and ended up with a spreadsheet that had 47 columns and no coherent narrative. The stakeholder asked one specific question at the end: "Did conversion go up?" I didn't have a clean answer. The whole engagement slipped by two weeks because I hadn't defined the scope upfront. A proper plan forces you to define the question before you start. It saves time. Not always, but usually.

The Components That Actually Matter

Every plan I write contains roughly the same sections. Some are more important than others. Most junior analysts pad their plans with unnecessary detail. Don't do that. 1. Objective statement. One to three sentences. What decision will this analysis inform? Not what you're interested in. What decision. If you can't name the decision, you don't have a real objective yet. Go back and ask the person who paid for this work. 2. Research questions. Break the objective into numbered questions. Each one should be answerable with data. "Is the new design better?" is not a research question. It's a wish. "Does the variant B conversion rate differ from variant A by more than 1.5 percentage points at the 95% confidence level?" is answerable.

3. Data sources and variables. List every table, dataset, or API endpoint you'll pull from. Name the columns. Specify whether they're raw or aggregated. This is where most projects stall later, so get it right early. I once spent four days debugging a join because I assumed a column name was consistent across two schemas. It wasn't. The documentation was wrong. I wasted eight billable hours on that mistake. 4. Data cleaning steps. Write out the exact transformations. Null handling. Outlier rules. Date parsing. If something requires a judgment call, note it here and flag it for review. Don't leave it for later. Later you won't remember why you made that choice. 5. Analysis method. State the statistical test, model, or framework you'll use. Justify it briefly. If you're doing a t-test, say why. If you're using a regression, say what you're controlling for. I've seen plans where the method section just said "we'll see what the data shows." That's not a method. That's avoidance.

6. Success criteria. Define what constitutes a meaningful result before you run the analysis. This isn't about p-hacking. It's about not getting seduced by noise. Set your thresholds. Minimum detectable effect. Confidence level. Sample size requirement. If the result doesn't meet these, it doesn't meet these. Simple. 7. Timeline and deliverables. When will each phase be done? What does the final output look like? A slide deck? A notebook? A dashboard? Stakeholders care about this more than they'll ever admit.

A Concrete Data Analysis Plan Example

Here's a stripped-down example from a recent project. Not all of it will apply to your situation, but the structure is repeatable. Objective: Determine whether introducing a one-click reorder button increases repeat purchase frequency among existing customers over a 60-day observation window. Research questions:

— Does the treatment group show a statistically significant increase in repeat purchases compared to control? — Is the effect larger for customers with three or more prior purchases? — Does the effect decay after day 30?

Data sources: — Postgres table: orders (order_id, user_id, created_at, amount) — Postgres table: order_events (event_type, timestamp, user_id)

— Segment tracking: page_view events filtered to /checkout path Variables: — Primary: number of repeat orders per user in 60 days

— Secondary: average order value, time to first repeat purchase — Covariates: prior purchase count, device type, acquisition channel Cleaning steps:

— Remove test accounts (user_id starting with TEST_) — Exclude users with fewer than 5 days of activity post-exposure — Handle null amounts by dropping rows where amount is NULL

— Dateparse all timestamps to UTC Analysis method: — Two-sample t-test for primary outcome

— Logistic regression for secondary question with interaction term for prior purchase count — Survival analysis (Kaplan-Meier) for time-to-repeat-purchase decay Success criteria:

— Minimum detectable effect: 3% relative lift — Confidence level: 95% — Power: 80%

— Required sample: approximately 12,400 users per group This took me about 45 minutes to write. The actual analysis took three weeks. The plan prevented at least two mid-project pivots and saved me from explaining to management why we'd been running an underpowered test for ten days without realizing it.

What Most People Get Wrong

The biggest mistake is treating the plan as a formality. It isn't. It's a constraint. Every good constraint reduces the space of possible mistakes. Another mistake is underestimating data quality work. A plan that assumes your data is clean is a plan that will fail. I now budget at least 30% of total project time for cleaning and validation. Sometimes more. If your source system is fragmented or logging is inconsistent, it can be 60%. There's also a blind spot around analysis paralysis. Some teams write a plan so detailed that they treat it as a contract instead of a guide. When the data reveals something unexpected, they ignore it because it wasn't in the plan. That's wrong. The plan should shape your initial approach, not replace your curiosity. Flag unexpected findings separately. Document them. Don't bury them because they weren't on the checklist.

Tools You Can Use

I write my plans in markdown files stored alongside the project repo. Not because they need to be flashy, but because version control lets you track changes. If you change your halfway through, that should be visible. Not hidden in an email thread. Some people use Notion. Some use Confluence. Some use Word. The tool doesn't matter. The discipline does. The discipline is writing it down before you start, revisiting it when you hit roadblocks, and updating it when you deviate. For a downloadable template, most organizations don't need anything custom. A two-page document with the seven sections above is enough. If you want a starter, I keep a minimal template at github.com/agnes-tmp/analysis-plan-template. It's plain markdown. Takes about ten minutes to adapt to your project.

When a Plan Won't Help

There are cases where a formal plan adds friction without proportionate benefit. Exploratory analysis, one-off dashboard requests, ad-hoc data pulls for meetings—these don't need a full plan. A bullet list of questions and data sources is sufficient. Plans matter most when the stakes are high, the timeline is long, or multiple people are involved. If you're the only person running the analysis and it takes three hours, you don't need a document. If you're managing a team, a budget, and a deadline, the plan is insurance. It won't prevent every problem. It won't fix bad data. It won't stop stakeholders from changing their minds. But it will make the path clearer when things go wrong, which they will.