How to Build Prescreen Assessment Questions That Actually Filter Candidates
I spent three years running technical hiring for a fintech startup, and the biggest headache was always the same: candidates who aced the phone screen fell apart the moment they saw real code. We wasted about forty hours a week on people who could talk a good game but couldn't write a function that handled edge cases. The fix wasn't better interviewers. It was better Prescreen Assessment Questions. Most teams get this wrong by starting with definition questions. You know the type — "What is polymorphism?" or "Explain the difference between REST and GraphQL." These questions don't predict job performance. They predict whether someone memorized a blog post the night before. I learned this the hard way when a candidate who nailed every definition question turned out to have never written a production API endpoint. She'd taken a certification course and regurgitated the material. We hired her anyway because she passed our old prescreen. She lasted six weeks.
What Are Prescreen Assessment Questions, Really
At their core, prescreen questions are binary filters designed to separate people who can do the job from people who can't, before you invest time in a full interview cycle. The "prescreen" part means they happen early — usually before a recruiter call or right after an application. The "assessment" part means they test actual capability, not knowledge recall. And the "questions" part is a bit of a misnomer, because the best prescreens aren't questions at all. They're tasks. A proper prescreen task looks like this: give the candidate a small, realistic piece of work that mirrors something they'd actually do on the job, set a reasonable time limit, and evaluate the output against a rubric. For a backend engineer, that might be writing a simple API endpoint with input validation and error handling. For a data analyst, it could be cleaning a messy CSV and producing three summary charts. For a project manager, maybe drafting a risk register for a hypothetical product launch. The key word is realistic. If the task has nothing to do with the actual work, you're just testing whether someone can solve puzzles under pressure. That's a different skill, and it correlates poorly with job performance unless the job literally involves solving puzzles all day.
How to Design Questions That Work
Start by listing the top five things your team actually does in a typical week. Not the exciting stuff. The bread-and-butter work. If your engineers spend most of their time writing SQL queries and debugging production incidents, don't give candidates a system design problem about building a distributed cache. They'll perform well on the prescreen and still struggle with day one. Here's a concrete example from my experience. We were hiring for a mid-level Python role at a payments company. The bread-and-butter work was writing idempotent API handlers, dealing with retry logic, and handling edge cases where third-party payment providers returned unexpected error codes. Our prescreen task asked candidates to write a function that processed a payment, handled three specific error scenarios, and logged everything correctly. We gave them ninety minutes. Eighty percent of applicants either didn't submit anything or produced code that failed on the second edge case. The twenty percent who passed all had prior experience with payment APIs. We hired from that group. Four out of five are still with us two years later. The rubric matters just as much as the task. Define what "pass" looks like before you hand out the assignment. For the payment function above, the rubric included: handles valid input correctly, returns appropriate error codes for each failure mode, doesn't double-charge on retries, and includes basic logging. Nothing fancy. Just the baseline expectations for someone who would be writing this code every day.
Get the Full Details
I should mention the timing issue that trips up most teams. Ninety minutes is usually the sweet spot for individual contributor roles. Less than that and you're testing whether someone can rush, not whether they can do the work. More than that and you're asking candidates to do a day's worth of unpaid labor, which is both unethical and a legal risk in some jurisdictions. If your task consistently takes more than ninety minutes for someone at the target skill level, it's too long.
Common Pitfalls and How to Avoid Them
The most common mistake is making the prescreen too hard. I've seen teams use take-home projects that required setting up a full development environment, integrating with external APIs, and deploying to cloud infrastructure. These aren't prescreens. They're second interviews disguised as filters. A candidate might fail because they don't have access to a $200 MacBook Pro, not because they lack the skills. This disproportionately screens out people from lower-income backgrounds and creates a homogenous hiring pool. Another pitfall is vague instructions. "Build something useful with APIs" sounds open-ended and creative, but it gives candidates no signal about what you actually care about. One candidate built a weather dashboard. Another built a crypto price tracker. A third built a habit tracker. None of them demonstrated the specific skills the role required. Your instructions should be tight enough that every submission tests the same capabilities, but loose enough that candidates can show their working style. Here's a specific edge case I ran into that I still think about. We once had a candidate who submitted a prescreen for a frontend role that was brilliant but used a framework we didn't support — Vue instead of React. The code was clean, the UI was polished, and she'd clearly put in significant effort. We rejected her because she didn't follow the tech stack specification. Six months later, we realized we'd missed someone who could have learned React in two weeks, while someone else we hired who knew React cold couldn't write responsive layouts to save their life. The lesson: clarify whether the task tests tool familiarity or fundamental skill, and weight your evaluation accordingly. In our case, the rubric should have been "can you build a functional UI component," not "does this use React."
There's also the problem of candidates using AI tools during the prescreen. This isn't hypothetical anymore. LLMs can write competent code, generate documentation, and even explain their reasoning. Some teams treat this as cheating and ban AI outright. Others embrace it and test whether candidates can direct AI tools effectively. Neither approach is wrong, but you need to be explicit about which rule applies and design your evaluation around it. If AI use is allowed, include a follow-up live session where the candidate explains their submission line by line. If AI is banned, state that clearly in the instructions and accept that some candidates will find workarounds anyway.
When Prescreen Assessment Questions Don't Work
Let me be blunt about the limitations. Prescreens are terrible at predicting cultural fit, long-term growth, or how someone handles ambiguity. They're also biased toward candidates who have had recent practice with that specific type of task. A senior engineer who hasn't written code in six months due to parental leave might fail a coding prescreen but perform exceptionally well once back on the job. A career changer with strong fundamentals might struggle with a task that assumes domain familiarity. If your hiring process relies entirely on prescreens, you'll miss good candidates and hire mediocre ones who game the system. The solution is to use prescreens as one signal among many, not the deciding factor. A well-designed prescreen might cut your interview pipeline from twenty candidates to five. It shouldn't be the final arbiter of who gets hired. There are also roles where prescreens simply don't apply. Sales positions, creative directions, executive leadership — these jobs depend on soft skills, strategic thinking, and interpersonal dynamics that a written task can't capture. For these roles, use structured interviews, portfolio reviews, or practical exercises conducted in real time. Don't force a prescreen onto a process where it doesn't belong just because it worked for engineering.
One more thing worth noting: prescreen completion rates are usually lower than you expect. Even when the task is short and well-designed, only sixty to seventy percent of candidates will submit something. Some drop off because the instructions weren't clear. Others realize mid-task that the role isn't what they expected and self-select out. A few just don't have time. Don't treat low completion rates as a sign that your prescreen is broken. It's just how hiring works. If you want higher completion, make the first touchpoint clearer and the task easier to understand before someone commits to it.
Practical Steps to Get Started
Start small. Pick one role where you're currently struggling — maybe the conversion rate from application to offer is abysmal, or new hires are underperforming after three months. Design a single prescreen task for that role, run it on the next five candidates, and evaluate the results against your rubric. Don't try to build a comprehensive prescreen program across all teams at once. That approach usually fails because you don't have enough data to know what works. Share the task with a few people outside your team before using it. Ask them to complete it and report where they got stuck, what was unclear, and how long it took. If three out of five test runners fail on the same edge case, your rubric or instructions are probably flawed. Fix it before you roll it out to actual candidates. Track your prescreen results over time. If the pass rate is consistently below ten percent, the task is too hard or your rubric is too strict. If it's above ninety percent, the task is too easy or you're not evaluating carefully enough. The target range is usually thirty to fifty percent for individual contributor roles, though this varies by seniority and market conditions.
Finally, don't treat your prescreen as a static asset. Every six months, review the submissions from the past quarter and look for patterns. Are certain edge cases consistently tripping people up? Is there a particular mistake that keeps appearing? Use those insights to refine the task, the instructions, or the rubric. The best prescreens evolve based on real data, not team opinions.
