Multiple-choice items are easy to write badly and hard to write well. A weak item rewards test-wise candidates, punishes careful readers, or simply measures nothing. A strong one separates people who have the skill from people who do not, and it does so in under a minute.

This article sets out a method subject matter experts can follow, a worked example of a flawed item and its fix, and a short review routine to run before any item goes live.

The parts of an item

Every multiple-choice item has three parts, and each has a job.

  • Stem. The question or problem. It should carry the whole task, so a strong candidate could answer it without seeing the options.
  • Key. The one correct answer. It must be defensible against a reference, not just the writer’s memory.
  • Distractors. The wrong options. Each should be plausible to someone who lacks the skill and clearly wrong to someone who has it.

When an item fails, it is usually because one part is doing another part’s job. A vague stem pushes the real question into the options. A key that is only “most correct” turns the item into an argument.

Start with one objective

Before writing a word, tag the item to one line of your blueprint. One item, one objective. If an item touches two objectives, a wrong answer cannot tell you which one the candidate missed.

Then decide the cognitive level. Recall items ask a candidate to remember a fact. Application items ask them to use the fact in a situation. Analysis items ask them to judge between options in a realistic case. Most certification blueprints call for more application than recall, so most items should start with a short scenario.

A useful test: could someone pass this item by memorizing a sentence from a textbook? If yes, it is a recall item, whatever the blueprint tag says.

Write the stem as a question

Stems work best as complete questions. Incomplete sentences that end in a blank invite grammatical cues and force candidates to reread each option against the stem.

Guidelines most item writing programs follow:

  • Put the scenario first and the question last.
  • Include only the detail the candidate needs. Extra detail tests reading, not skill.
  • State the question positively. If you must use “not” or “except,” set it in bold so nobody misses it.
  • Keep the reading level at or below what the job requires.
  • Avoid names, places and idioms that some candidates will find unfamiliar.

Covering the options and asking an expert to answer the stem alone is a fast check. If they can, the stem is carrying the task.

Build distractors from real errors

Good distractors are not random wrong answers. They are the specific mistakes that people without the skill actually make.

Sources for plausible errors:

  • Common misconceptions your instructors or supervisors see
  • Steps done in the wrong order
  • A correct rule applied to the wrong situation
  • A calculation with one predictable slip, such as a unit error
  • Answers that would be right under an older standard or code

How many options? Many programs use four, and some use three. The number matters less than whether each distractor actually attracts people. Two strong distractors beat three where one is filler that nobody chooses. Your item statistics will show which distractors pull their weight after the first window.

Cues that give the answer away

Test-wise candidates look for patterns. Remove them.

  • Length. The key is often the longest, most qualified option. Keep options similar in length.
  • Grammar. If the stem ends in “an,” only options starting with a vowel fit.
  • Absolutes. Options with “always” or “never” are usually wrong, and candidates know it.
  • Repetition. A word in the stem that appears only in the key points straight at it.
  • Position. Writers tend to hide the key in the middle positions. Let the platform randomize, or balance positions by hand.
  • Catch-all options. “All of the above” can be chosen by spotting two true options. “None of the above” tells you a candidate knows what is wrong, not what is right.

Example: fixing a flawed item

Here is an illustrative item for a food safety certification, first as a writer might draft it.

Stem: Cooling food properly is important because

  • A. it tastes better
  • B. bacteria can grow rapidly in the temperature danger zone if food is cooled too slowly, which can lead to foodborne illness
  • C. it is the law
  • D. all of the above

Problems: the stem is an incomplete sentence with no task. The key (B) is far longer than the rest. “All of the above” competes with the key, since C is arguably true. And the item tests recall of a reason, not the skill of cooling food.

Here is the revised item.

Stem: A cook has a 4-gallon pot of chili at 140°F at 2 p.m. Which method will cool it to 70°F within two hours?

  • A. Leave the covered pot on the counter, then refrigerate it at 4 p.m.
  • B. Divide the chili into shallow pans and place them in the walk-in cooler
  • C. Place the covered pot in the walk-in cooler right away
  • D. Leave the pot uncovered on the counter with a fan on it

The stem now poses a realistic task and can be answered without the options. Each distractor reflects a real error: waiting too long, trusting the cooler to do the work alone, and a half measure that feels active. Options are similar in length. The key is checked against the program’s reference code, and the item is tagged to the cooling objective at the application level.

Review before it goes live

Writers miss their own flaws. Every item should pass at least three reviews by someone other than its writer.

  1. Content review. A second expert confirms the key against a cited reference and checks that each distractor is clearly wrong.
  2. Editorial review. An editor checks reading level, grammar, consistent style and cues.
  3. Bias and accessibility review. A panel checks for unfamiliar contexts, stereotypes, and anything that a screen reader or a candidate with a disability would struggle with, such as information carried only by color.

Our item review checklist lists the checks in each group and is free to copy into your own process.

Log each review against the item: who reviewed it, when, and what changed. That record is part of your content evidence, and it answers the question an appeal panel will eventually ask.

Check the item after the window

Review stops a lot, but candidates find the rest. After the first window, look at three numbers for every new item:

  • Difficulty. The proportion of candidates who chose the key.
  • Discrimination. Whether high scorers chose the key more often than low scorers.
  • Distractor counts. How many people chose each wrong option, split by high and low scorers.

A distractor that nobody chooses is doing nothing. A distractor that high scorers choose more than the key may mean the key is wrong. Our article on item analysis basics walks through each signal with examples.

New items often run as unscored pretest items first, so their statistics can be checked before they count toward anyone’s result.

What to do this week

  • Pick ten live items and cover the options. Can an expert answer each stem alone?
  • Check the same ten for length cues and catch-all options.
  • Tag every item to one blueprint line and one cognitive level.
  • Ask instructors or supervisors for the five errors they see most often, and save them as distractor sources.
  • Pull distractor counts for your last form and list any option chosen by almost nobody.

Where we can help

Writing items is a skill experts learn fastest together. Our Item Writing Workshop trains your subject matter experts in item construction, bias review and accessibility review, then has them write and review items for your own blueprint with a facilitator in the room. It is indicative from $5,500.

For programs with a larger rebuild ahead, our psychometrics and exam development service covers the whole path from job analysis to technical report, with every item’s history kept in the bank.