The 2010 Revision of the TOEIC® Speaking Test Bradley Irwin and Peter Nagy |
It is the broader trait of communicative competence, not specific individual skills, that is critical in most academic and workplace settings and of most interest to users of tests like the TOEFL and TOEIC. It is important, however, to test for each of these four skills individually because each is a critical aspect of communicative competence. Furthermore, direct evidence of specific individual skills can provide at least indirect evidence of other skills. (p. 9)By expanding the test to include speaking and writing, Educational Testing Service (ETS) hoped allow test takers to better demonstrate their holistic language skills.
[ p. 10 ]
More specifically, Powers et al. (2009, p. 1) state that the new speaking component has the following three objectives:[ p. 11 ]
Table 1Claim | Claim 1: Successful test takers can generate language intelligible to native and proficient nonnative English speakers | Claim 2: Successful test takers can select appropriate language to carry out routine social and occupational interaction | Claim 3: Successful test takers can create connected, sustained discourse appropriate to the typical workplace | |||
Task Type | 1: Read Aloud | 2: Describe a Picture | 3: Respond to Questions (Survey type) | 4: Questions about Schedule or Agenda | 5: Problem Solving | 6: Expressing Opinions |
Question # and Content |
Questions 1 - 2 Two Texts |
Question 3 One Photo |
Questions 4 - 6 Two Short Questions One Long Question |
Questions 7 - 9 Two Basic Info. Questions One Summary Question |
Question 10 Respond to a dilemma |
Question 11 State an Opinion about a Paired Choice |
Time Allotment | Prep. Time: 45 seconds |
Prep. Time: 30 seconds |
Prep. Time: Nil |
Prep. Time: Nil |
Prep. Time: 30 seconds |
Prep. Time: 15 seconds |
Response Time: 45 Seconds |
Response Time: 45 Seconds |
Response Time: 15/15/30 seconds |
Response Time: 15/15/30 seconds |
Response Time: 60 seconds |
Response Time: 60 seconds |
|
Evaluation Criteria | Pronunciation Intonation |
Grammar, Vocabulary Cohesion, Criteria from Tasks 1 and 2 |
Relevance of content Completeness of content Criteria from Tasks 1 and 2 |
Criteria from Tasks 1-3 | Criteria from Tasks 1-3 | Criteria from Tasks 1-3 |
Evaluation Scale | 0 - 3 | 0 - 3 | 0 - 3 | 0 - 3 | 0 - 5 | 0 - 5 |
Integrated (listen & speak) | Integrated (read, listen, speak) | Integrated (long listen & speak) | Independent |
[ p. 12 ]
Now let us examine the TOEIC Speaking Test according to the previously mentioned Bachman and Palmer framework.[ p. 13 ]
reported by ETS for multiple test-takers of the speaking test (Liao & Qu, 2010, p. 6). The raw score coefficients are listed as .80 (agreement between first and second sittings of the test), .80 (agreement between second and third sittings), and .79 (agreement between third and fourth sittings). The median time interval between each test-retest were 63 days between the first and second sittings, 45 days between the second and third sittings, and 28 days between the third and fourth sittings. However, the actual score level correlation coefficients were somewhat lower at .76, .77, and .74. For information on the sample size and length of time between sittings please refer to Table 2.Sitting | Sample Size | Median Time Interval Between Tests | Raw Score Correlation Coefficient | Score Level Correlation Coefficient |
1 - 2 | 16,867 | 63 days | 0.80 | 0.76 |
2 - 3 | 5,129 | 45 days | 0.80 | 0.77 |
3 - 4 | 1,923 | 28 days | 0.79 | 0.74 |
[ p. 14 ]
A further concern is using examinee self-reported can-do lists as indicators of test validity because of the disparity between one's perceived and actual abilities. Even the ETS-produced report on the TOEIC Speaking Test's validity could not establish the soundness of using test-takers' self-reports as valid measures (Powers et al., 2010, p. 14). For example, 2% of examinees who scored the lowest level (between 0-50 out of 200) felt that they could serve as interpreters for managers in business negotiations while only 47% of examinees who scored the highest level (between 190-200 out of 200) felt they could perform the same role. This particular task, the most difficult listed on the can-do report, produced a very low correlation (.32) with actual test scores. Overall, the correlation between speaking can-do reports and actual TOEIC Speaking Test scores was somewhat weak at .54 (Powers et al., 2010, p. 5).[ p. 15 ]
[ p. 16 ]
Mislevy, R. J., Steinberg, L. S., & Almond, R. G. (2002). Design and analysis in task-based language assessment. Language Testing, 19, (4) 477-496. doi: 10.1191/0265532202lt241oaWhen you go on vacation to a foreign country there are many things you must consider. Many people travel to foreign countries to relax, but some vacations can be stressful. Properly preparing yourself for an upcoming vacation may help you avoid stressful situations. The kind of trip you take can also help reduce stress. Some people enjoy going to beach resorts to relieve tension. These types of places usually have a variety of activities to choose from. Diving, snorkeling, swimming and sunbathing are all common activities at beach resorts. Don't forget to bring your camera with you when you take a trip. |
[ p. 17 ]
Questions 4 - 6: Question response
Question 4: How often do you watch movies? |
|
[ p. 18 ]
Question 7: What is the cost of attendance?Question 10: Solution proposal
Question 8: Can you please tell me when the seminars start and finish?
Question 9: I may be somewhat late in the morning. Can you please tell me about the morning seminars?
|
Question: (Narrator): Some people argue that environmental issues such as global warming and climate change are caused by human action. What is your opinion about environmental issues like climate change and global warming? Please provide reasons for your opinion. |
[ p. 19 ]