Building a Working Security Awareness Quiz Program
Most companies build these quizzes wrong. They copy generic questions from a vendor template, blast them out once a year, and call it compliance. Two months later, nobody remembers what they learned, and the phishing simulation rate goes straight back up. I built a quiz program from scratch at my last place, watched it fail, rebuilt it, and now it actually works. Here is how. Start by understanding what you are actually testing. The two main categories are knowledge recall and behavioral response. Knowledge recall asks "what is the minimum password length?" Behavioral response asks "what do you do when you get an email like this?" The second one is what matters. A person can memorize policy and still click a suspicious link the next day. Your questions need to simulate decisions, not vocabulary tests. I spent about three weeks just mapping out scenarios before writing a single question. I pulled our actual phishing campaign data from the previous twelve months and looked at which emails our staff kept falling for. The top three were: fake IT password reset requests, urgent CEO wire transfer emails, and delivery notification spam. Those became the core of my quiz bank.
Question format that works: Show them a real screenshot of a phishing email, not a description. People recognize patterns visually. Put the actual sender address, subject line, and one suspicious element. Ask what they should do. Give four options where only one is correct, but make the wrong answers plausible enough that someone skimming would pick them. If every distractor is obviously wrong, the question is useless. Here is a realistic example from my own quiz bank: Question: You receive an email from support@secureit-now.com with the subject "Action Required: Your VPN Credentials Expired." It includes a link to update your password and sounds urgent. What do you do?
A) Click the link immediately and change your password so you are not locked out B) Reply to the email asking for clarification C) Forward the email to the IT security team and delete it
Get the Full Details
D) Go to the company VPN portal manually by typing the URL into your browser The correct answer is D. C is a common instinct but forwarding potentially malicious content to an internal address is a vector. B engages with the attacker. A is exactly what they want. The explanation that follows the quiz should walk through why each wrong answer is dangerous, not just say "the right answer is D." One problem I ran into that took me weeks to solve: people started memorizing answers instead of learning behavior. After the second quarterly quiz, our phishing click rate went down but only because everyone had seen similar questions before. The quiz became a recall test rather than a training tool. My workaround was switching to a scenario-generation model. Instead of reusing the same questions, I built a smaller set of core scenarios with randomized elements — different sender names, different urgency levels, different pretext themes — so the underlying behavior being tested stayed the same but the exact question changed every time.
This cut our question bank maintenance time from about 40 hours per quarter down to maybe six, because we only needed to maintain the scenario templates instead of writing dozens of full questions. The tradeoff is that it takes more upfront work to design solid templates, and the randomization can occasionally produce edge cases that are too easy or too ambiguous. I caught that by reviewing a sample of generated quizzes before they went live. About one in twenty would have a distractor that was clearly incorrect due to the randomization, and those got flagged and fixed. For question distribution, here is what actually showed measurable improvement in our environment over six months: - Phishing identification: 30% of questions
- Password and credential hygiene: 20% - Social engineering (phone, in-person, chat): 15% - Data handling and classification: 15%
- Physical security and clean desk: 10% - Incident reporting procedures: 10% Any other balance is fine, but if you spend half your quiz on password complexity rules, you are not preparing people for what actually happens. Most breaches in mid-size companies come from phishing or social engineering, not someone writing their password on a sticky note. Tailor the weight to your risk profile.
Scoring and consequences: I've seen companies make the mistake of failing people who score below a threshold, which just makes everyone game the system or share answers. Instead, use low scores as a flag for optional remedial training. Score above 80 percent and you are good. Score between 60 and 80 and you get a brief follow-up module on your weak areas. Below 60 and you go through the full refresher. Nobody gets penalized on their record for a low score. The goal is behavior change, not embarrassment. The feedback after each question is the most important part of the quiz. That is where the actual learning happens. Keep it to two or three sentences. Don't paste the entire security policy into the explanation. Tell them what they missed and why it matters in plain terms. For the actual quiz platform, most security awareness tools like KnowBe4, Proofpoint, or Cofense have built-in quiz builders that handle the randomization and scoring automatically. If you are doing this manually through something like Google Forms or a custom LMS, budget extra time for the randomization logic. Building it yourself is possible but fragile. I tried it once and spent more time maintaining the quiz system than actually improving the content.
A counter-intuitive thing about these quizzes: shorter is better. A 15-minute quiz once a quarter produces better retention than a 45-minute one every month. People tune out after about 20 minutes regardless of how good the content is. Get the key scenarios in, give solid explanations, and stop. Fatigue is real and it undermines whatever you are trying to teach. The one area where quiz-based training completely fails is for roles with specialized security responsibilities. Developers, sysadmins, and anyone handling classified data need role-specific training that goes well beyond a multiple-choice quiz. A quiz can reinforce knowledge, but it cannot replace hands-on training for people who actually configure firewalls or deploy code. Budget for that separately. If you need a starting point for question templates, NIST SP 800-50 has publicly available guidance on security awareness training content. It is dry and not formatted as a quiz, but it covers the domains you need to address. Many organizations also draw questions from the CIS Controls framework. Neither is a ready-made quiz you can just import, but both give you the structure to build one that actually fits your environment.
