Qualitative Literacy A Guide To Evaluating Ethnographic And Interview Research

How to actually read qualitative research without getting lost in jargon

Most people who come across ethnographic studies or interview-based research don't know what they're looking at. They see pages of quotes from participants and assume the researcher is just collecting stories. That's not the job. The job is to figure out whether the researcher actually earned whatever conclusion they're selling, and that requires a specific set of skills that barely gets taught outside of graduate programs. I spent roughly eight years working with qualitative data before I stopped counting. The first time I properly evaluated a research report instead of just writing one, I realized how many published studies in our field couldn't survive five minutes of scrutiny. That's not because the researchers were careless. It's because the evaluation framework itself isn't intuitive. You can't just apply quantitative logic to qualitative work and expect it to hold up.

Qualitative Literacy A Guide To Evaluating Ethnographic And Interview Research

The core of qualitative literacy is understanding that rigor doesn't look like the same things in qualitative work as it does in quantitative. When you evaluate ethnographic research, you're looking for evidence of systematic engagement with the data, not statistical significance. When you evaluate interview research, you're looking for transparency about how questions were asked and how participants were selected, not sample size. Let me walk through what actually matters when you're reading one of these studies critically. Positionality and reflexivity are the first things I check. Every qualitative researcher carries bias. The question isn't whether they have it, it's whether they've accounted for it. A well-written study will describe who the researcher is, what their relationship is to the community or topic being studied, and how that relationship might shape what they saw and heard. If a researcher says they had no influence on the data, that's a red flag. They always had influence. The ones worth reading admit it and explain what they did to mitigate its effects.

I once reviewed a report from a consulting firm that claimed to have conducted in-depth interviews with hospital administrators. The methodology section was one paragraph long. The researchers described their own demographic details in a single sentence at the end. What they missed was a critical detail: two of the three interviewers had previously worked for competing healthcare organizations. That background absolutely shaped how they interpreted resistance and pushback from participants. The findings weren't wrong, but the framing assumed that the administrators' language was about system efficiency when it could have been about institutional loyalty. Without acknowledging those professional histories, the analysis lacked credibility. Sampling strategy is the second thing I look at. Qualitative research doesn't use random sampling. It uses purposive, theoretical, or snowball sampling, and the difference matters enormously. Purposive sampling means you selected participants because they had specific experiences relevant to your research question. Theoretical sampling means you recruited new participants as your analysis evolved and you identified gaps in your emerging theory. Snowball sampling means existing participants referred you to others. Each approach has different validity implications. If a study says "we interviewed 47 people" without explaining how those 47 were chosen, I can't evaluate the trustworthiness of the findings. Forty-seven people selected through convenience at a single conference site gives you dramatically less generalizable insight than forty-seven people recruited through theoretical sampling across three different organizations over eight months. The number alone tells you almost nothing.

Data saturation is a term that gets thrown around loosely. In practice, saturation means you've stopped finding new themes in your data. Not that you've found all the themes possible in the world, but that your current dataset isn't generating additional categories worth tracking. The problem is that many researchers claim saturation after fifteen interviews when they really just ran out of time or budget. I've seen legitimate studies continue collecting until they'd done sixty-plus interviews because the population was heterogeneous enough that new patterns kept emerging. There's no universal cutoff. The claim needs to be backed up by showing that later interviews reinforced themes already established rather than introducing fundamentally new ones. Here's something beginners almost never catch: negative case analysis. The strongest qualitative studies actively search for data that contradicts their emerging conclusions. If your study is about why patients trust certain doctors and every single interview confirms that trust, you haven't done good work. You've done confirmatory work. Good researchers spend as much time tracking down the cases that don't fit as the ones that do. When you read a study, look for whether the authors address contradictory evidence. If they don't mention it at all, either they didn't encounter it or they chose to ignore it. Both are problems, though the second is worse. Thick description is another concept that gets misused constantly. Thick description doesn't mean long descriptions. It means descriptions that capture enough context that someone reading them can understand why a particular behavior or statement made sense within its setting. A thin description says someone refused to participate in a meeting. A thick description explains that the meeting was held in a space where participants had historically been disciplined for speaking up, that the facilitator had a reputation for redirecting dissent, and that the refusal was consistent with patterns observed in prior meetings. The difference determines whether a reader can judge whether the researcher's interpretation is plausible.

When I evaluate interview research specifically, I pay close attention to the interview protocol itself. How were questions ordered? Were they open-ended or leading? Did the interviewer probe deeper when something unexpected came up, or did they stick rigidly to the script? The way questions are phrased directly shapes what data comes back. A question like "Did you find the training helpful?" generates a very different response than "Tell me about your experience with the training." The first pushes toward agreement. The second lets the participant define what helpful means. Studies that report using semistructured interviews but then present data that clearly came from closed questions are hiding something important. Audit trails separate serious qualitative work from opinion pieces. An audit trail is a documented record of every analytical decision made during the research process. Why did you code this excerpt this way? Why did you merge these two themes? Why did you drop that participant's data from the final analysis? When a study includes an audit trail or supplementary materials that let another researcher follow your analytical path, it dramatically increases confidence in the findings. Most published studies skip this entirely. That doesn't automatically mean they're bad, but it means you're taking the author's word for their process without a way to verify it. One common pitfall in evaluating this kind of research is assuming that more data equals better research. That's not true. A study with thirty richly analyzed interviews that demonstrates clear analytical rigor will often produce more trustworthy findings than a study with one hundred superficial interviews analyzed with a generic coding framework. The depth of engagement with the data matters more than the volume. I've seen dissertations with hundreds of pages of raw transcripts and weak analytical work. I've also seen twenty-page reports that did more with less because the researcher knew exactly what they were looking for and how to extract it.

Credibility, transferability, dependability, and confirmability are the four criteria originally proposed by Lincoln and Guba for evaluating qualitative research. They're useful as a checklist even though they're not perfect. Credibility means the findings reflect reality as closely as possible given the constraints. Transferability means readers can determine whether the findings apply to their own context. Dependability means the research process is logical and traceable. Confirmability means the findings are shaped by the participants rather than researcher bias. When I read a qualitative study, I run it through these four filters. If a study scores poorly on confirmability, for example, I'm skeptical about whether the conclusions are genuinely emergent or artificially imposed. There are limits to what qualitative literacy can do. Sometimes the research design is fundamentally flawed and no amount of critical reading will rescue it. I've encountered studies where the researcher spent three weeks in a community but never learned the local language, conducted all interviews through a translator who was also a colleague of community leaders, and then presented the findings as if they captured authentic community perspectives. No evaluation framework fixes that. The only honest response is to set it aside and look elsewhere. Another limitation is that qualitative research simply cannot answer certain types of questions. If you need to know what percentage of a population holds a particular view, ethnographic or interview-based methods are the wrong tool. They excel at explaining why people hold views, how those views operate in practice, and what meanings people attach to them. They don't generalize statistically. Some researchers try to make them do that work anyway, and the results are usually misleading.

The practical takeaway is straightforward. Read qualitative research the way you'd read any other claim: check the evidence, examine the methods, look for what's missing, and decide whether the conclusions are supported by what's actually on the page. Don't accept authority because of a publication's name. Don't dismiss findings because the methods feel less familiar than statistics. And don't assume that because a study feels compelling or emotionally resonant, it's methodologically sound. Those are different things entirely.