A work sample test asks a candidate to do a piece of the actual job, under set conditions, and scores the result against a rubric. A dispatcher takes a mock call. A bookkeeper reconciles a short ledger. A technician finds a fault on a training panel.
Done well, it shows the hiring team the work itself. Done in a hurry, it becomes an unscored chat with props. This article lays out a two-week plan for one role, ten working days, with a team that can give it a few hours a day. Treat the timing as a common pattern, not a promise; complex roles take longer.
Why work samples are worth it
Sackett, Zhang, Berry and Lievens (Journal of Applied Psychology, 2022) found that work sample tests had a mean operational validity of .33, a correlation between test scores and later job performance. In their review, that placed work samples above cognitive ability (.31) and below structured interviews (.42). The SIOP summary of the review explains the full ranking.
Work samples have practical value too. Candidates tend to see them as fair because the link to the job is obvious. Hiring managers see real work, not a description of it.
Hiring assessments are covered by employment law. Have your employment counsel review the test, the rubric and how scores are used before any candidate sits it.
Who you need
Keep the team small and named.
- A hiring manager who owns the decision the test supports.
- Two or three experienced staff who do the job well and can describe good work in detail.
- A facilitator who knows rubric design and keeps the process on schedule.
- Someone from HR to handle scheduling, accommodations and records.
Avoid putting only the newest or only the most senior staff on the panel. A mix gives a truer picture of what the job asks on an ordinary day.
Days 1 and 2: the task
Start with the job, not the test. Ask the experienced staff to list the tasks they do in a typical week and rate each on two things: how often it happens and how much damage a mistake does.
Pick the one or two tasks that score high on both and that a new hire must do from day one. Leave out tasks that are taught on the job in the first month. Testing those measures prior experience, not readiness.
Then cut the task down to fit. Common practice is to keep a work sample between 30 and 90 minutes. Long enough to see real work, short enough that candidates and raters can manage it.
Days 3 and 4: the materials
Write the task exactly as a candidate will see it: the instructions, the materials, the time limit, and what they hand in.
- Use realistic materials with names, numbers and details changed.
- Give every candidate the same version, or two versions of equal difficulty.
- State what tools and references are allowed.
- Say how long they have and how they will know time is nearly up.
- Describe what “finished” means, so candidates are not left guessing.
Ask one staff member who did not help write the task to read the instructions and say back what they would do. Any confusion here will be worse with candidates.
Days 5 and 6: the rubric
The rubric is what turns a task into a test. Without it, scores reflect each rater’s taste.
Break the task into four to six scoring dimensions, each tied to something that matters on the job: accuracy, safety, order of steps, communication, time. For each dimension, describe what weak, adequate and strong work looks like, in observable terms.
- Weak: “Misses the second fault; note does not say what was tested.”
- Adequate: “Finds both faults; note lists tests in order, with readings.”
- Strong: “Finds both faults, states the likely cause and flags a related safety check.”
Decide how dimensions combine. A total score is simple. Some programs also set a must-pass dimension, often safety, that a candidate cannot make up for elsewhere. Write the rule down now, not after scoring the first candidate. Our article on writing rubrics that score consistently goes deeper on anchors.
Day 7: rater training
Raters need to read the rubric the same way. Bring them together for two to three hours.
Walk through each dimension and its anchors. Then have every rater score the same two or three sample responses alone, using work produced by staff. Compare scores. Where raters differ by more than one level, talk through why and tighten the anchor wording.
Agreement between raters is called inter-rater reliability. You do not need a formal statistic on day seven. You need raters who, given the same work, land on the same level most of the time.
Days 8 and 9: staff pilot
Before candidates see the test, run it with four to six current staff at different levels of experience. Time them. Score them with the trained raters. Ask each one what was unclear.
Look for three things. Did the task take about as long as planned? Did stronger staff score higher than newer staff? Did any dimension get the same score from everyone, which suggests it is not telling raters anything?
Fix what the pilot shows. Most fixes are small: a clearer instruction, a tighter time limit, a rewritten anchor.
Day 10: rules and records
Finish with the decisions that make the test consistent from one candidate to the next.
- How scores feed the hiring decision, and what weight the test carries.
- Who rates, and whether two raters score each candidate.
- How accommodations are requested and configured, such as extra time or assistive technology.
- What candidates are told before the test, including the time and allowed tools.
- How materials are stored so the task does not circulate.
- How long scores and work are kept, and who can see them.
Then send the package to counsel for review before the first live candidate.
Example: a dispatcher work sample
Example scenario. Say a county emergency communications center hires dispatchers twice a year. Its current process tests typing speed and a general aptitude test.
The panel picks one task: take a recorded mock call, enter the incident in a training version of the dispatch system and assign a unit, in 20 minutes. The rubric has five dimensions: address captured correctly, priority code, questions asked in order, unit assigned, and calm, clear speech. Address and priority are must-pass.
Three supervisors train on four recorded staff responses and find they disagree on “questions in order”. They rewrite the anchor to list the required first three questions by name. Five current dispatchers pilot it; two finish in 14 minutes, one runs over. The center keeps 20 minutes and adds a two-minute warning. This is an illustration, not a client account.
After the first round
Once real candidates have taken the test, check how it behaves. Look at the spread of scores on each dimension, how often two raters agreed, and how long candidates took. After a few months, ask hiring managers how new hires who scored high are doing on the tasks the test covers.
Programs commonly monitor pass rates by group at each step, with counsel, and review the task if differences appear. Validity is built over time. As the Standards for Educational and Psychological Testing put it, it is “the degree to which evidence and theory support the interpretations of test scores for proposed uses of tests.”
What to do this week
- Name the role and the hiring manager who owns it.
- Ask three experienced staff to list their weekly tasks and rate frequency and consequence.
- Circle the one task a new hire must do well from day one.
- Draft the candidate instructions in one page.
- Book a two-hour rater session for next week.
Our Role Readiness Assessment program (indicative, from $4,500 per role) builds the work sample, rubric and rater training with your staff. It sits within our skill evaluation systems, which keep tasks, scores and rater notes on one record. Our assessment glossary defines rubric, inter-rater reliability and kappa for anyone new to the terms.
