Article appearing in Shiken 17.1 (May 2013) pp. 11-22.
Author: Aaron Olaf Batty
Keio University
Lancaster University
Abstract:
This article explains investigating over- and under-confidence on tests and test items using the method developed by Paek et al. (2008) for use with Rasch and other IRT measures. The data for this demonstration originated from a study of 199 Japanese high school and university students, investigating their knowledge of a number of special uses of verbs of utterance in English. The paper provides practical information on the calculations necessary for the use of the Paek et al. method under the Rasch model and the interpretation of the results. Finally, the same data are scaled with the two-parameter IRT model and the Paek et al. method is applied for comparison.
Keywords: confidence, accuracy, discrepancy, overconfidence, underconfidence, Rasch, two-parameter IRT