Article appearing in Shiken 19.1 (April 2015) pp. 16-31.
Author: Jim Smiley
Tohoku Bunka Gakuen University
Abstract:
Educators have utilized Classical Test Theory (CTT) when developing instruments for measuring and assessing pedagogic data. Results derived from standard CTT analysis methods offer valuable insights into the effectiveness of language assessment tools. Tests undergo a series of steps running from initial draft production through test trialing to test revision. Such instruments produced using this method can be shown to be more internally consistent and deliver more valid results than tests that are written ad hoc and informed by intuitive rationale. More recently, the Rasch model has gained a following among test developers as an alternative procedure in refining testing vehicles. CTT contrasts with the Rasch model in a number of key areas, differences that, when utilized in the analysis of a test, result in the production of a more internally valid test. This article questions the need for a materials developer to change to Rasch given that the learning curve is steep considering the additional investment of the time necessary to become proficient using Rasch. The conclusion is that Rasch data provides very detailed information that is sine qua non for long-term test instrument refinement and materials development, and that CTT data may be enough to begin the test of the test.
Keywords: Rasch modelling, Classical Test Theory, comparisons, test analysis