Article appearing in SHIKEN 27.1 (June 2023) pp.3-14
Article DOI: https://doi.org/10.37546/JALTSIG.TEVAL27.1-1
By Jean-Pierre J. Richard
The University of Nagano
Abstract
The effects of two years of online classes, due to measures put in place to prevent the spread of COVID-19 among university students in Japan, remains largely unknown. This study investigated four cohorts of students (N = 854) at a prefectural university in regional Japan who completed the Test of English for International Communication, Listening and Reading (TOEIC L&R) at the start and end of their two-year required English language program. TOEIC gain scores were examined in relation to expected regression toward the means and to the standard error of difference. These data were also analyzed using a difference-in-differences quasi-experimental design. Key findings indicated that all four cohorts made large, significant gains. Members of the cohort that experienced two years of online learning, however, made significantly fewer gains compared with the other three cohorts on Listening, and significantly fewer gains compared with its successor cohort on Reading. Overall, the findings suggest that the two years online had a deleterious effect with regard to TOEIC L&R scores. However, several important limitations were addressed. Most importantly, the quality of the online TOEIC L&R when it was introduced in 2020 leaves some doubt about the conclusions drawn.
Keywords: TOEIC, online learning, COVID-19, regression to the mean, standard error of difference, difference-in-differences, pretest-posttest data, Japanese university students