Article appearing in Shiken 17.2 (December 2013) pp. 23-26.
Author: James Sick
American Language Institute, Tokyo Center
New York University School of Continuing and Professional Studies
Opening paragraph:
The previous installment of this series dealt with how many-facet Rasch analysis (MFRA) can be used to adjust for differences in rater severity when measures are constructed from subjective judgments. This installment addresses the related issue of rater agreement and inter-rater reliability from the perspective of classical test theory (CTT) and Rasch measurement theory (RMT).