How to Build Actually Useful Comprehension Questions
The first thing most people get wrong is that reading comprehension questions aren't just about testing whether someone read the book. They're testing whether someone can navigate it, argue with it, and use it. I've written hundreds of these over the years for everything from dense technical manuals to sprawling fiction, and the structure is always the same even if the execution changes wildly depending on the material. Start with a passage. Not the whole chapter. A section. Two or three pages maximum if you want questions that are actually answerable without re-reading the entire book. The question difficulty should map to how someone will use the text afterward. If they're a student preparing for an exam, you need literal and inferential questions mixed together. If they're a professional looking for actionable takeaways, you skip the plot-summary level entirely and go straight to application and evaluation. The four-tier framework covers this well enough: literal recall (what does the text explicitly state), inference (what does the text imply but not say), analysis (how do the ideas connect or contradict), and evaluation (is the argument sound or useful). Most people stop at literal and inference. That's lazy question design. The tier three and four questions are where actual comprehension shows up because they force the reader to hold multiple ideas in tension at once.
I remember spending a week on a comprehension set for a 400-page systems engineering textbook. The client wanted questions for a certification exam, and they kept sending back drafts that were too easy. The problem was structural: the book used the same terminology across five different chapters with subtly different meanings each time. I had to build questions specifically targeting those terminological collisions, like asking someone to distinguish between "latency" as defined in the networking section versus the database section. That's not something you catch by skimming. You have to flag every term and note its contextual shifts. Took me three days just for the glossary cross-referencing before I wrote a single question.
The Practical Workflow
Here's what I actually do when I'm building these out. First, I mark up the text as I read it. I underline claims, circle definitions, and note where the author makes a transition from describing something to arguing about it. That second step is critical because transition points are where inference questions are born. An author might describe three case studies and then say "this suggests" without ever spelling out the logical link. That gap is a question waiting to happen. Then I write the literal questions first. Keep them simple and unambiguous. There should be one clearly correct answer. If a literal question has two defensible answers, the question is flawed and needs to be rewritten or cut. You'd be surprised how many people include ambiguous literal questions without realizing it. After the literal set, I move to inference. This is where most question writers stumble because they confuse inference with speculation. Inference has to be grounded in textual evidence. The answer can't be stated directly, but it has to be the most reasonable conclusion you can draw from what's actually there. A good check is: could someone reasonably disagree with your answer while still being textually accurate? If yes, rewrite the question.
Get the Full Details
Analysis questions require you to map relationships between ideas. Causal chains, contrasting viewpoints within the same text, structural organization choices. These questions tend to have longer stems and more complex answer choices. That's fine. They're testing higher-order thinking, and the formatting should reflect that. Evaluation questions are the hardest to write well because they require you to bring outside knowledge into the mix without making the question about that outside knowledge. The passage should contain enough internal logic that a careful reader can assess the argument on its own terms. A common mistake is writing an evaluation question where the correct answer depends on facts the reader wouldn't reasonably know. Don't do that. Keep the evaluation grounded in what the text provides or in universally accepted criteria.
Answer Choices That Actually Work
The distractors matter as much as the correct answer. Every wrong option should represent a plausible misunderstanding someone might have. A random wrong answer tells you nothing about the reader's comprehension level. A wrong answer that reflects a specific cognitive error—the kind of error the passage is designed to prevent—tells you exactly where the teaching failed. For example, if a passage argues that correlation doesn't imply causation, one of your distractors should be the exact confusion the passage is trying to address: "The data shows X causes Y because they occur together." That wrong answer reveals the reader fell for the same trap the author warned against. Better than any random alternative. I keep distractor templates in a spreadsheet. When I'm working quickly, I pull from those instead of inventing from scratch. It saves time and maintains consistency across a full question set. The template covers common reasoning errors: conflating necessity and sufficiency, assuming directionality in bidirectional relationships, ignoring confounding variables, and overgeneralizing from a single example.
Common Mistakes I See Repeatedly
One major issue is question stacking. That's when a single question tries to test two or three different comprehension tasks at once. The reader has to correctly identify a concept AND trace its causal chain AND evaluate a counterargument, all in one item. These questions produce noisy data because you can't tell which skill the reader actually failed on. Split them into separate questions. Another is over-reliance on direct quotation matches. Writers love questions where the answer is a near-exact phrase lifted from the text. That tests vocabulary recognition more than comprehension. Replace some of those with paraphrase-based questions that require the reader to restate the idea in their own words before selecting the answer. Finally, the context problem. If the book is fictional, literal questions about plot details can feel hollow. Readers often know what happens but can't explain why. In those cases, shift your emphasis toward character motivation, thematic patterns, and narrative structure. The same framework applies, just applied to different literary elements instead of factual claims.
When This Approach Fails
Reading comprehension questions don't work for every type of text. Poetry resists this format because the primary value is in the language itself, not in propositional content. Trying to turn "The Road Not Taken" into multiple-choice questions about theme misses the point almost entirely. Same goes for highly experiential texts—memoirs, travel narratives, instructional guides where the knowledge is procedural rather than declarative. For those, you need different assessment methods: discussion prompts, performance tasks, reflective essays. If you're generating these questions in bulk for a course or training program, you'll also hit diminishing returns after about two dozen high-quality items. More questions beyond that point usually means lower quality unless you're building a large-item bank for adaptive testing. In that case, write in batches of ten, review each batch against the others for overlap, and retire any question that doesn't add new diagnostic information. The whole process for a moderate-length nonfiction book (around 250 pages) typically takes me three to four hours for a solid set of twenty-five to thirty questions covering all four tiers. That's with careful marking, deliberate answer choice construction, and a pass where I rewrite anything that feels too obvious. Rushed versions come out in under two hours and show it.
What matters more than the number of questions is whether they actually measure something. A set of ten well-designed questions that reveal gaps in understanding is worth more than fifty that mostly confirm the reader paid attention.
