A blueprint is the weighted content outline every form is built from. Its weights decide how many items each domain gets, which decides what the total score mostly reflects. If safety is 10 percent of the blueprint, safety is about 10 percent of every pass or fail decision.
Many programs inherited their weights from a committee meeting years ago. This article shows how to set them from job data instead, with a worked example you can copy.
What a weight actually decides
Weights look like an administrative detail. They are a claim about the job.
Example: say a 120-item exam gives 15 percent to documentation and 10 percent to diagnostics. A candidate who is strong at paperwork and weak at diagnosis can pass more easily than the reverse. If the job is mostly diagnosis, the exam is rewarding the wrong strength.
Weights also drive cost. Every percentage point is items to write, review, pretest and replace. Overweighting a narrow domain means writing many items about the same few facts, which raises exposure and makes forms easier to memorize.
Where weights should come from
For certification and licensing exams, the usual source is a job task analysis. A panel of practitioners lists the tasks and knowledge the job involves. A larger group of practitioners then rates each task in a survey.
Common rating scales:
- Importance. How important is this task to safe, effective practice? (1 to 5)
- Frequency. How often does a practitioner at this level perform it? (1 to 5, from never to daily)
- Criticality. How serious is the harm if it is done wrong? (1 to 5)
Programs combine these in different ways. Some multiply importance by frequency. Some add criticality as a third factor or use it to set a floor. There is no single correct formula, but there must be a written one, chosen before the survey results come in.
For course and district exams, the source is the curriculum: instructional time, standards emphasis and course objectives. The method below works the same way with those inputs.
Example: from ratings to items
Here is an illustrative calculation for a residential HVAC certification with five domains and a 120-item scored form. The ratings are survey means from practitioners, and the program has chosen importance multiplied by frequency as its formula.
| Domain | Importance | Frequency | Product | Weight |
|---|---|---|---|---|
| Safety | 4.6 | 4.2 | 19.32 | 27.7% |
| Installation | 4.1 | 3.8 | 15.58 | 22.3% |
| Diagnostics | 4.4 | 3.1 | 13.64 | 19.5% |
| Maintenance | 3.5 | 3.9 | 13.65 | 19.5% |
| Codes and documentation | 3.2 | 2.4 | 7.68 | 11.0% |
| Total | 69.87 | 100% |
Each weight is the domain’s product divided by the total. Safety is 19.32 divided by 69.87, or 27.7 percent.
To turn weights into items, multiply by 120:
- Safety: 33.18
- Installation: 26.76
- Diagnostics: 23.43
- Maintenance: 23.44
- Codes and documentation: 13.19
Round with a written rule
Fractions of items do not exist, so you round. Rounding each number to the nearest whole gives 33, 27, 23, 23 and 13, which sums to 119. One item is missing.
A common rule is the largest remainder method. Round every domain down, then give the leftover items to the domains with the largest fractions. Here the fractions after rounding down are .18, .76, .43, .44 and .19. Installation (.76) and maintenance (.44) get the two leftover items:
- Safety: 33
- Installation: 27
- Diagnostics: 23
- Maintenance: 24
- Codes and documentation: 13
That sums to 120. The rule matters less than writing it down and applying it the same way every time.
Let a panel adjust, on record
Survey math gives a starting point, not a final answer. A panel of practitioners should review the draft weights and can adjust them, provided each change has a written reason.
Reasons panels commonly give:
- Criticality. Diagnostics is done less often than maintenance but errors cause serious harm, so the panel moves it up.
- Coverage. A domain with few rated tasks would need many items on the same narrow content, so the panel caps it.
- Level. A task rated highly by senior practitioners is beyond entry level, so the panel moves it to a different credential.
In the example, the panel might move diagnostics from 23 to 26 items and take three from installation, noting that diagnostic errors carry higher criticality ratings. That decision, its reason and the vote go into the blueprint record. Without the record, a future reviewer sees an unexplained gap between survey and blueprint.
Add a cognitive level axis
Content weights say what is tested. A second axis says how. Many blueprints split each domain into recall, application and analysis.
Example: the safety domain’s 33 items might be planned as 8 recall, 18 application and 7 analysis. That plan tells item writers what to write and stops the bank from filling with easy recall items, which are the fastest kind to produce.
Keep the cognitive split simple. Three levels are enough for most programs, and experts agree on them more readily than on finer scales.
Watch the small domains
Small domains raise two questions.
First, subscores. If you plan to report a score for codes and documentation, 13 items is a thin basis. Many programs report small domains only as broad bands, or not at all, and say so in the score report.
Second, form-to-form stability. With few items, swapping two of them can change the domain’s difficulty noticeably. Many programs set a tolerance for form assembly, such as each domain within one item of its target, and have the platform check every form against it.
When to revisit the weights
Weights age as the job changes. Common triggers for a new analysis:
- A major change in codes, regulations, equipment or practice
- A new scope of practice or a new credential level
- A pattern of appeals or expert complaints about one domain
- A scheduled cycle; many programs plan a fresh job task analysis every five to seven years as common practice
Between full analyses, a short review by a practitioner panel can confirm that the weights still fit.
Mistakes that skew weights
A few patterns show up again and again when programs review old blueprints.
- Weighting by item supply. A domain gets 30 percent because the bank happens to hold many items on it, not because the job calls for it.
- Weighting by committee voice. The most senior panelist’s specialty grows a little every cycle.
- Counting pretest items. Unscored items are mixed into the weight counts, so the scored form drifts from the blueprint.
- Changing the formula midstream. The method is picked after the survey results are in, which makes the choice look like a way to reach a preferred answer.
- Weights with no rounding rule. Two staff members build two forms and round differently, so the forms are not equivalent.
Each of these is fixable once it is visible. The fix is usually the same: write the rule down, apply it to every form, and keep the record with the blueprint.
One more check is worth running. Compare the blueprint with what your item bank actually holds. If a domain needs 24 items a form and the bank holds 30 usable items for it, two forms will share most of their content. That is a writing target for the next workshop, not a reason to lower the weight.
What to do this week
- Find your current weights. Locate the blueprint and the record of how its weights were set.
- Check the source. If there is no survey or curriculum basis, note that as a gap in your validity file.
- Compare forms. Count items per domain on your last two forms. Do they match the blueprint?
- Write the rules. Draft your weighting formula, rounding rule and form tolerance on one page.
- List the triggers. Note anything in the field that has changed since the blueprint was set.
Our free exam blueprint guide walks through seven steps from content outline to sample blueprint table, and it pairs well with the example above.
Where we can help
When weights need a fresh basis, our Job Task Analysis program runs the practitioner panel, the survey and the weighting meeting, and delivers a blueprint with every decision documented. It is indicative from $14,000.
Once the blueprint is set, the online testing platform tags every item to its domain and cognitive level and checks each assembled form against your targets, so the weights you agreed are the weights candidates see.
