Many exams pass at 70 percent because the last exam did. Nobody can say what a 70 means, and the number stays the same when the items get harder or easier.

A cut score is the score that separates pass from fail. A defensible one is set by a documented study, so you can show a candidate, a board or a reviewer how the line was drawn and by whom. We help programs build that record. Whether it holds up in any particular challenge is for a court or accreditor to decide, not us.

Why a fixed percentage fails

A fixed percentage ties the decision to the items, not to the candidate. Swap ten easy items for ten hard ones and the same person drops below the line without knowing any less.

It also skips the question that matters: how much does someone need to know to do this job, or move to this course, safely? That is a judgment, and it should come from people who know the work, made with a method and written down.

The Standards for Educational and Psychological Testing (AERA, APA and NCME, 2014) define validity as “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests.” A pass or fail is an interpretation. The cut score study is a large part of the evidence behind it.

Start with the borderline candidate

Every common method begins with the same step: describing the minimally qualified candidate. This is the person who just barely deserves to pass. Not the average candidate, and not the expert.

A good description is concrete. It names tasks, not virtues.

  • “Can calculate a medication dose from a written order without a reference table.”
  • “Recognizes when a load calculation needs an engineer’s review, but may not complete it.”
  • “Knows the reporting deadline for a client complaint and the form it goes on.”

Panels usually spend the first two to three hours of a study on this description. It feels slow. It is the part that makes the ratings that follow mean the same thing across panelists.

Pick a method that fits

There are several established standard setting methods. Most programs choose among a few.

  • Modified Angoff. Each panelist estimates, item by item, the share of borderline candidates who would answer correctly. The cut is the sum of those estimates, averaged across panelists.
  • Bookmark. Items are ordered from easiest to hardest by real response data. Panelists place a bookmark where a borderline candidate would stop answering reliably.
  • Borderline group. Raters who observe candidates in a practical exam flag those who are borderline, and the cut sits at the typical score of that group.
  • Ebel. Panelists sort items by relevance and difficulty, then judge how many in each group a borderline candidate gets right.

Angoff is the most familiar for multiple-choice certification exams. Bookmark suits programs with enough response data to order items. Borderline group methods suit performance exams with trained raters. Our assessment glossary defines each method, along with the terms that follow.

Who sits on the panel

The panel is your evidence that the judgment came from the field. Common practice is to recruit enough subject matter experts that no single voice dominates, often eight to twelve, and to vary them by setting, region, years of practice and specialty.

Leave out anyone with a conflict, such as people who teach prep courses for the exam or who wrote a large share of the items under review. Record each panelist’s background and any declared conflicts. If you are accredited or plan to be, this links directly to your impartiality record.

A trained facilitator runs the session, usually a psychometrician. Their job is method and process, not content. They should never tell the panel where the line should be.

How a study runs

A one-day or two-day study for a single exam commonly follows this order.

  1. Orientation. The purpose of the exam, the blueprint and the method, in plain terms.
  2. Take the exam. Panelists sit the form, timed, without the key. This resets their sense of difficulty.
  3. Define the borderline candidate. As above, written and agreed.
  4. Practice round. Five to ten items rated and discussed, to calibrate.
  5. Round one ratings. Each panelist rates every item alone.
  6. Discussion. The facilitator shows where ratings spread widely, and panelists explain their reasons.
  7. Impact data. Panelists see what share of recent candidates would pass at the round one cut.
  8. Round two ratings. Panelists may revise. Many will move a little; some will not move at all.
  9. Evaluation forms. Each panelist records their confidence in the process and the result.

The cut that comes out is a recommendation. The decision belongs to the program’s governing body, which reviews it with the full record.

Example: a 100-item Angoff study

Example, with invented numbers for illustration only. A licensing exam has 100 scored items. Ten panelists rate each item with the share of borderline candidates likely to get it right.

Item 14, a routine safety rule, averages 0.85 across panelists. Item 62, a rare calculation, averages 0.40. Adding the average rating for all 100 items gives 67.4. Round two, after discussion and impact data, moves the total to 68.1.

The recommended raw cut is 68 out of 100. The program’s board reviews the panel record, the impact data and the standard error of measurement, then adopts 68 for the current form. If the next form is harder, the cut for that form is adjusted through equating, the statistical process that keeps one form’s 68 equal to another form’s score, rather than by a new guess.

The standard error and policy choices

Every score carries some measurement error. The standard error of measurement estimates how much a candidate’s score might vary across equivalent forms.

Some boards adjust the panel’s cut by a small amount in one direction. Lowering it limits the chance of failing a qualified candidate. Raising it limits the chance of passing an unqualified one. Which error matters more is a policy decision for the board, not a statistical one, and the reason should be written into the minutes.

What to keep on file

The study only helps if it is documented. A complete record usually includes:

  • The borderline candidate description, as agreed.
  • Panelist names, roles, backgrounds and conflict declarations.
  • The method, the materials and the facilitator’s script.
  • Every rating from every round.
  • Impact data shown to the panel.
  • Panelist evaluation forms.
  • The board’s decision, its reasoning and the date.

This becomes a chapter in your technical report. It is also what you reach for when a candidate asks how the pass mark was set.

When to revisit the cut

A cut score is not permanent. Common triggers for a new study or a formal review include a new blueprint after a job task analysis, a change in the exam’s length or format, a change in who the exam is for, and item statistics that show the forms have drifted. Between those events, equating keeps the standard steady across forms.

What to do this week

  1. Find out how your current cut was set. If the answer is “we have always used 70”, write that down too.
  2. Draft a one-paragraph description of your borderline candidate with three concrete tasks.
  3. List ten possible panelists across settings and regions, and note any conflicts.
  4. Pull pass rates for the last four windows, so you have impact data ready.
  5. Check that your blueprint is current. If it is more than five years old, a Job Task Analysis may need to come first.

Our Standard Setting Study program (indicative, from $7,500) runs the panel, the method and the record for one exam. If you are not sure the exam itself is ready, a Test Health Check (indicative, from $1,900 per exam) reviews the item statistics first, with a written report in ten business days.