Article appearing in Shiken 26.2 (December 2022) pp. 19-38;
Article DOI: https://doi.org/10.37546/JALTSIG.TEVAL26.2-2
By Jonathan Harrison1 and Ruth Vanbaelen2
1. Nihon University, College of Science and Technology
2. University of Tsukuba
Abstract
Can-do statements have been integrated for decades into research and marketing strategies related to standardized tests of English for non-native language learners. For instance, while ETS indicated that TOEIC can-do statements are specifically for “decision making in organizations,” the Council of Europe created user-friendly statements that all stakeholders, including test takers, can use and understand. This study discusses a partial replication of a large-scale validity study with the purpose of developing a scale which can be used by examinees in combination with the TOEIC SW can-do statements for self-assessment. As part of an extracurricular program, 23 graduate-level EAL university students took the TOEIC SW and speaking and writing can-do self-assessment surveys. Results indicated that both self-assessment scales used in this study had moderate to strong correlations with TOEIC SW scores (r = 0.62–0.73) and TOEIC proficiency levels (r = 0.47–0.66). Self-assessment results using a Level Mean Average (LMA) scale correlated significantly with both speaking and writing test results and produced slightly higher percentages of test takers self-assessing within the true score zones of 1 SEM. For the most part, self-assessment predictive accuracy using these scales and true score zones was 52–57% for speaking and 43% for writing. Other findings were that EAL learners tended to underestimate their scores, and they more accurately self-assessed speaking than writing. If LMA scale accuracy at a particular level can be verified in future studies, claims could be made that TOEIC SW can-do statements have value for examinees.
Keywords: Assessment, can-do statements, English language learning, self-assessment, speaking, TOEIC, predictive accuracy, writing