Article appearing in Shiken 19.1 (April 2015) pp. 3-15.
Authors: Rie Koizumi1, Yo In'nami2, Junichi Azuma3, Keiko Asano1, Toshie Agawa1, and Derek Eberl1
1. Juntendo University
2. Chuo University
3. Kobe Gakuin University
Abstract:
Regression to the mean (RTM) and the standard error of difference (SED) are two artifacts commonly observed in pretest– posttest designs, but they are rarely addressed in practice. We examined whether second language (L2) learners' change in scores reflected change in their L2 proficiency, by investigating whether their actual scores exceeded those that considered RTM and SED; we did so by using pretest–posttest data of the Test of English as a Foreign Language Institutional Testing Program (TOEFL ITP) at a Japanese university across three years. We found moderate degrees of RTM, but also found that more than one-third (33.33-46.03%) of students increased their scores beyond RTM and the SED. We discuss the importance of considering RTM and SED in analyzing pretest–posttest data.
Keywords: considering errors in practice, pretest–posttest data, TOEFL ITP, Japanese university students