What Level Of Evidence Is A Mixed Methods Study

Understanding Evidence Levels in Mixed Methods Research

Mixed methods studies combine quantitative and qualitative approaches in a single investigation. When you are grading the level of evidence for these studies, the answer depends on what framework you are using and what the study actually did. The most common hierarchy of evidence frameworks—like those from the Oxford Centre for Evidence-Based Medicine or the Joanna Briggs Institute—treat mixed methods differently depending on their design. A well-conducted mixed methods study with a strong integrated design often sits at Level 2 or Level 1b in some frameworks. But this is not a hard rule.

What Level Of Evidence Is A Mixed Methods Study

It really comes down to how the two components relate to each other. If you have a randomized controlled trial with a concurrent embedded qualitative strand that helps explain the mechanism, you might see it classified as Level 1b. If the qualitative part dominates and the quantitative part is just descriptive, it drops closer to Level 3 or 4. I worked on a systematic review last year where we had to appraise twenty-seven mixed methods papers. The reviewers kept arguing about whether a particular study was Level 2 or Level 3. The problem was that the study used a convergent parallel design but the qualitative sample was only twelve people from a single ward. The quantitative side had decent power, but the integration was superficial. We ended up classifying it as Level 3 because the qualitative component could not support strong causal claims. Most guidelines will tell you that mixed methods studies occupy a middle tier of evidence. They are stronger than purely qualitative descriptive studies but generally weaker than a well-done randomized trial. The integration quality matters more than the mere fact that both methods exist in the same paper.

How Different Frameworks Handle Mixed Methods

The Oxford CEBM 2011 guidelines place mixed methods studies under Level 2 when they use prospective cohort designs with good follow-up. But the 2022 update softened this stance and acknowledged that study quality varies enormously within the mixed methods category. Joanna Briggs Institute takes a different approach. They do not assign a single level of evidence to mixed methods. Instead, they use qualitative evidence appraisal tools for the qualitative component and quantitative tools for the quantitative component. The synthesis then depends on the quality of each part. Campbell Collaboration guidelines sit somewhere in between. They recognize that mixed methods can provide high-confidence findings when the qualitative and quantitative strands corroborate each other. But they also note that convergence is rare and when it happens, the evidence strength increases.

The problem I keep running into is that journals and funding bodies want a single letter or number. They want to know if a study is Level 1, 2, or 3. Mixed methods studies frustrate this need because they are inherently multi-faceted. Forcing them into one tier usually means losing important information about what the study actually contributes.

Common Pitfalls When Grading Mixed Methods Evidence

Beginners tend to grade mixed methods studies too high because they assume that combining methods automatically creates stronger evidence. This is incorrect. A poorly conducted mixed methods study with weak integration can provide less useful evidence than a straightforward randomized controlled trial. Another mistake is treating the qualitative component as decoration. Some studies add interviews or focus groups after analyzing survey data, but never use those findings to interpret or explain the quantitative results. In my experience, these studies should be graded no higher than the quantitative component alone would warrant. I encountered a particularly frustrating case where a study claimed Level 1b evidence status. The authors had conducted a randomized trial with an embedded qualitative interview component. The qualitative interviews happened three months after the trial ended, and only four participants agreed to be interviewed. The integration section was one paragraph stating that participants were satisfied with the intervention. This does not constitute meaningful mixed methods integration. We downgraded it to Level 3 after peer review.

The worst scenario involves retrospective data linking. Some researchers take existing qualitative data and quantitative datasets from different sources and attempt to merge them post hoc. This is not true mixed methods research. It is data recycling with a fancy label. These studies should never receive high evidence grades.

When Mixed Methods Actually Strengthen Evidence

There are specific scenarios where mixed methods designs genuinely improve the evidence base. The most valuable use case is when the qualitative component explains mechanisms that quantitative data alone cannot clarify. If you have a trial showing that an intervention works but you do not know why, adding a process evaluation with interviews can fill that gap. Another legitimate use is early-stage theory development. Before you can design a rigorous randomized trial, you often need qualitative exploration to understand the context, identify barriers, and develop appropriate measures. Studies in this phase belong to lower evidence tiers, but they are essential for building the foundation for higher-level research. I have found that the strongest mixed methods evidence comes from studies where both components are designed together from the outset. The quantitative and qualitative questions should align. The sampling strategy should be justified for both parts. The integration should happen at multiple points during analysis, not just in a discussion section tacked onto the end.

When integration is deep and methodological rigor is high in both strands, a mixed methods study can approach Level 1 evidence in terms of practical applicability. The statistical confidence interval might be wider than a pure RCT, but the contextual understanding and implementation relevance can compensate. This trade-off is often overlooked by evidence hierarchies that prioritize internal validity above all else.

Practical Guidance for Your Own Work

If you are planning a mixed methods study and want it to carry strong evidence weight, prioritize integration over volume. Ten well-conducted interviews that directly address your research questions matter more than fifty superficial ones. A clear protocol that specifies how and when you will integrate findings prevents the common problem of disjointed components. Document your integration decisions transparently. Reviewers and graders need to see exactly how you used the qualitative findings to interpret the quantitative results, or vice versa. A table mapping each research question to its quantitative and qualitative sources makes this much easier to evaluate. Be honest about limitations. Mixed methods studies are expensive and time-consuming. If your qualitative sample is small or your integration is partial, say so. Inflated evidence claims damage credibility more than admitting modest contribution. The evidence hierarchy exists to help clinicians and policymakers make decisions. Accurate grading serves that purpose better than optimistic inflation.

For a complete reference on how different organizations classify mixed methods evidence, you can download the JBI mixed methods appraisal tool from their website. It is free and updated regularly. The Campbell Collaboration also has guidance documents that address mixed methods specifically. Both are useful when you need to justify your evidence grading decisions in a systematic review or research proposal.