Psychometrics has its own vocabulary. These are the terms we use with clients, written so a program manager, a board member or an instructor can follow a technical report.
Thirty-two terms, A to V
- Accommodations
- Changes to how a test is given, such as extra time, breaks, a screen reader or a separate room, that remove a barrier without changing what the test measures. Each one should be approved and recorded before test day.
Where this shows up in our work - Angoff method
- A standard setting method in which subject matter experts estimate, item by item, the chance that a minimally competent candidate would answer correctly. The panel’s estimates are summed and averaged to recommend a cut score.
Where this shows up in our work - Beta testing
- Giving new items or a new form to real candidates before the scores count, to collect statistics and find problems. Items that perform poorly are revised or dropped before the live form is set.
Where this shows up in our work - Blueprint
- The weighted content outline every form is built from: the domains, the share of items in each, and often the thinking level and item types. Also called a test specification.
Where this shows up in our work - Bookmark method
- A standard setting method in which panelists read items ordered from easiest to hardest and place a bookmark where a minimally competent candidate stops being likely to answer correctly. The order comes from item response theory difficulty estimates.
Where this shows up in our work - Computerized adaptive testing (CAT)
- Delivery in which an algorithm picks each next item from the candidate’s answers so far, using item response theory. It can reach a precise score with fewer items but needs a large, calibrated item bank.
- Continuing education (CE)
- Learning that credential holders complete between renewals, counted in hours or credits against a published requirement.
Where this shows up in our work - Cronbach’s alpha
- An estimate of internal consistency reliability, from 0 to 1, based on how strongly the items on a form vary together. It rises with test length and with how varied the candidate group is, so read it alongside both.
- Cut score
- The score that separates pass from fail, set by a documented standard setting study rather than by habit. Also called a passing score or passing standard.
Where this shows up in our work - Differential item functioning (DIF)
- When candidates from different groups with the same overall ability have different chances of answering an item correctly. A DIF flag starts a content review; on its own it does not prove an item is biased.
- Discrimination
- How well an item separates candidates who know the material from those who do not. It is usually reported as a point-biserial correlation or an item response theory discrimination parameter.
Where this shows up in our work - Equating
- A statistical method that adjusts scores on different forms of the same exam so a given score means the same thing whichever form a candidate took.
Where this shows up in our work - Form
- One assembled version of an exam, with a fixed set of items and ordering rules. Programs rotate two or more forms built to the same blueprint.
Where this shows up in our work - Inter-rater reliability
- The degree to which two or more scorers give the same score to the same response. It is reported as percent agreement, kappa or a correlation.
Where this shows up in our work - Item bank
- The managed collection of every item a program owns, each stored with its content tags, review history, statistics and status.
Where this shows up in our work - Item difficulty
- The share of candidates who answer an item correctly, written as a p-value from 0 to 1. An item with a p-value of 0.90 is easy; one at 0.30 is hard.
Where this shows up in our work - Item exposure
- How often candidates have seen an item. Heavily exposed items are more likely to be shared, so programs rotate forms and retire items past a set limit.
Where this shows up in our work - Item response theory (IRT)
- A family of statistical models that give the probability of a correct answer from the candidate’s ability and the item’s properties, such as difficulty and discrimination. It underpins adaptive testing, on-the-fly forms and many equating designs.
Where this shows up in our work - Job task analysis (JTA)
- A structured study of what practitioners actually do, usually a panel workshop followed by a survey that rates tasks for importance and frequency. The results set the blueprint’s domains and weights.
Where this shows up in our work - Kappa
- A measure of agreement between two scorers that removes the agreement expected by chance. Weighted kappa gives partial credit when scorers differ by one point on an ordered rubric.
Where this shows up in our work - Learning Tools Interoperability (LTI)
- A 1EdTech standard that lets a learning management system launch an outside tool with single sign-on and receive grades back. Exampit supports LTI 1.3 for Canvas, Moodle, Blackboard and D2L Brightspace.
Where this shows up in our work - Linear on-the-fly testing (LOFT)
- Delivery in which each candidate gets a unique form, assembled from the item bank at the start of the session to match the blueprint and statistical targets. It spreads exposure across the bank while every form still follows the outline.
- Open Badges
- A 1EdTech standard for digital credentials that carry verifiable data about who earned what, from whom and when. Version 3.0 follows the W3C Verifiable Credentials model.
Where this shows up in our work - Point-biserial correlation
- The correlation between answering one item correctly and the total score. Positive values mean stronger candidates tend to get the item right; values near zero or below flag the item for review.
Where this shows up in our work - Proctoring
- Supervising a test session, in person or remotely, to confirm identity and to deter, detect and document conduct that breaks the exam rules.
Where this shows up in our work - Question and Test Interoperability (QTI)
- A 1EdTech standard for moving items and tests between systems with their content and scoring intact. Exampit imports and exports QTI 2.1 and 3.0.
Where this shows up in our work - Record and review
- A remote proctoring mode in which the session is recorded, software flags events, and a trained person reviews every flag afterward. At Exampit that review happens within two business days.
Where this shows up in our work - Recertification
- Renewing a credential at a set interval, by exam, continuing education or both, to confirm the holder is still competent.
Where this shows up in our work - Reliability
- The consistency of scores: how similar a candidate’s result would be on another form, on another day or with another scorer. Reliability is needed for validity but does not prove it.
Where this shows up in our work - Rubric
- A scoring guide for open responses that defines each score point with criteria and, ideally, sample answers at each level.
Where this shows up in our work - Standard error of measurement (SEM)
- An estimate, in score points, of how far an observed score is likely to sit from the candidate’s true score. It matters most for candidates just above or just below the cut score.
- Validity
- “The degree to which evidence and theory support the interpretations of test scores for proposed uses of tests,” in the words of the Standards for Educational and Psychological Testing (AERA, APA and NCME, 2014). It belongs to a use of a score, not to the test itself.
Where this shows up in our work
Hear these terms on your exam
Book a 30-minute demo and ask for a psychometrician to join.