Inter-scorer reliability refers to what?

• A. The degree of agreement between two or more scorers
• B. The stability of scores over time
• C. The accuracy of the scoring rubric
• D. The correlation between different tests

Answer: (D)

Inter-scorer reliability is about how consistently different people who score the same work rate it. When multiple scorers assign similar grades or categories to a response, agreement is high and reliability is strong. This matters because scoring can be subjective, so training raters, using clear rubrics, and calibrating scorers help ensure fairness and consistency across evaluations.

It’s not about whether scores stay the same when the same person takes a test again (that’s stability over time), nor about how accurate a scoring rubric is at reflecting true performance (that relates to rubric validity), nor about how scores from different tests relate to one another (that concerns relationships between instruments or validity across forms).