Shiken:JALT Testing & Evaluation SIG Newsletter
Vol. 3 No. 2 January 2000 (p. 2 - 7) [ISSN 1881-5537]
PDF PDF Version

University entrance examinations:
Strategies for creating positive washback
on English language teaching in Japan

James Dean Brown
University of Hawai'i at Manoa


The purpose of this article is to explore some of the ways the university entrance examinations in Japan could be used to foster positive washback effects on English language instruction. During the last twelve years, a great deal has been written about the quality and appropriateness of examinations in Japan.
As far back as 1987, the possibility of including listening tests on Japanese university English language entrance examinations became an issue (Brown and Christensen, 1987). Other issues that have arisen over the years include ways to improve the testing of false beginners (Brown, 1987), ways to improve the fit of tests to language programs (Brown, 1990), and ways that standardized test results are sometimes misinterpreted in Japan (Brown, 1993).
Beginning in 1995, the issues of examination hell and appropriateness of the university entrance examinations for testing English language skills in Japan became issues (Brown, 1995a, 1995b; Brown and Kay, 1995; Brown and Gorsuch, 1995). Also in 1995, Brown and Yamashita began publishing the results of their analyses of the English language parts of twenty-one university entrance examinations (including ten private, ten public, and the center exam) (Brown and Yamashita, 1995a, 1995c), studies that did not go entirely without criticism (see for instance, O'Sullivan, 1995, answered in Brown and Yamashita, 1995b).
In 1996, discussion of the issue of the English language parts of the Japanese university entrance examinations came to a head when Brown (1996a) delivered his plenary speech at the annual JALT Conference in 1995. Stapleton (1996) offered a reaction to some of Brown's points, arguing basically that Brown was ignorant of the Japanese perspective. Yoshida (1996a) started a series of articles when he offered the view that Brown was practicing "cultural imperialism" by ignoring important cultural differences between Japan and the United States. Brown (1996b) answered Yoshida's arguments point-by-point and Yoshida (1996b) gave a final response (because, perhaps mercifully, the Daily Yomiuri would not accept an additional response article by Brown). The most recent addition to the literature on this topic also came from the ever-persistent Brown (1998), in which he discussed the general effects of university entrance examinations on English language teaching in Japan.

[ p. 3 ]

In the articles above, a number of criticisms were leveled and problems were identified but also, a few solutions were offered. In this short paper, I hope to expand the solutions by summarizing some of the positive aspects of two other papers wherein I dealt with the washback effect in general (Brown, 1997) and the washback effect and its relationship to Japanese university English language entrance exams (Brown, 1998). Before doing that however, it would be useful to define the notion of washback.

Some Definitions of Washback

Shohamy, Donitsa-Schmidt and Ferman (1996) define washback as "the connections between testing and learning" (p. 298). Gates (1995) define washback simply as "the influence of testing on teaching and learning" (p. 101). Messick (1996) refers to washback as ". . . the extent to which the introduction and use of a test influences language teachers and learners to do things they would not otherwise do that promote or inhibit language learning" (p. 241). The washback effect clearly has to do with the effect of external testing on the teaching and learning processes in language classrooms. One example is the ways the university entrance examinations in Japan affect high school language teaching and learning.

Promoting Positive Washback

In searching the literature on washback I found that Hughes (1989), Heyneman and Ransom (1990), Shohamy (1992), Kellaghan and Greaney (1992), Bailey (1996), and Wall (1996) all provided lists of strategies for using the washback effect to positively influence language teaching. For more extensive discussion of these lists, see Brown, 1997, 1998. However, no two lists agreed on what those strategies should be or how they are related to each other. In the following outline, I attempt to summarize and organize the strategies proposed in the literature into four different categories that language educators in Japan can use to promote positive washback: test design strategies, test content strategies, logistical strategies, and interpretation strategies.
"I attempt to summarize and organize the strategies proposed in the literature into four different categories that language educators in Japan can use to promote positive washback: test design strategies, test content strategies, logistical strategies, and interpretation strategies."


[ p. 4 ]

A. Test design strategies
  1. Sample widely and unpredictably (Hughes, 1989)
  2. Design tests to be criterion-referenced (Hughes, 1989; Wall, 1996)
  3. Design the test to measure what the programs intend to teach (Bailey, 1996)
  4. Base the test on sound theoretical principles (Bailey, 1996)
  5. Base achievement tests on objectives (Hughes, 1989)
  6. Use direct testing (Hughes, 1989; Wall, 1996)
  7. Foster learner autonomy and self-assessment (Bailey, 1996)
B. Test content strategies
  1. Test the abilities whose development you want to encourage (Hughes, 1989)
  2. Use more open-ended items (as opposed to selected-response items like multiple choice) (Heyneman and Ransom, 1990)
  3. Make examinations reflect the full curriculum, not merely a limited aspect of it (Kellaghan and Gleaney, 1992)
  4. Assess higher-order cognitive skills to ensure they are taught (Heyneman and Ransom, 1990; Kellaghan and Greaney, 1992)
  5. Use a variety of examination formats, including written, oral, aural, and practical (Kellaghan and Greaney, 1992)
  6. Do not limit skills to be tested to academic areas (they should also relate to out-of-school tasks) (Kellaghan and Greaney, 1992)
  7. Use authentic tasks and texts (Bailey, 1996; Wall, 1996)
C. Logistical strategies
  1. Insure that test-takers, teachers, administrators, curriculum designers understand the purpose of the test (Bailey, 1996; Hughes, 1989)
  2. Make sure language learning goals are clear (Bailey, 1996)
  3. Where necessary, provide assistance to teachers to help them understand the tests (Hughes, 1989)
  4. Provide feedback to teachers and others so that meaningful change can be effected (Heynelman and Ransom, 1990; Shohamy, 1992)
  5. Provide detailed and timely feedback to schools on levels of pupils' performance and areas of difficulty in public examinations (Kellaghan and Greaney, 1992)
  6. Make sure teachers and administrators are involved in different phases of the testing process because they are the people who will have to make changes (Shohamy, 1992)
  7. Provide detailed score reporting (Bailey, 1996)
D. Interpretation strategies
  1. Make sure exam results are believable, credible, and fair to test takers and score users (Bailey, 1996)
  2. Consider factors other than teaching effort in evaluating published examination results and national rankings (Kellaghan and Greaney, 1992)
  3. Conduct predictive validity studies of public examinations (Kellaghan and Greaney, 1992)
  4. Improve the professional competence of examination authorities, especially in test design (Keuaghan and Greaney, 1992)
  5. Insure that each examination board has a research capacity (Kellaghan and Greaney, 1992)
  6. Have testing authorities work closely with curriculum organizations and with educational administrators (Kellaghan and Greaney, 1992)
  7. Develop regional professional networks to initiate exchange programs and to share common interests and concerns (Kellaghan and Greaney, 1992)

[ p. 5 ]

Discussion
“. . . the majority of these strategies would probably only work if there was comprehensive teamwork and collaboration between the university examination writers and the instructors who teach high school English.”


In view of the points made in the literature on positive washback, we have to ask ourselves how such test design, test content, logistical, and interpretation strategies could be applied to the university entrance examinations in Japan as well as who would implement them. Clearly, some of the strategies listed above would have to be the responsibility of the people who design and write the university English language entrance exams, but the majority of these strategies would probably only work if there was comprehensive teamwork and collaboration between the university examination writers and the instructors who teach high school English. Such large-scale cooperation to achieve positive washback from the entrance examinations in Japan could probably only be organized by the Ministry of Education [Sports, Science and Technology] or the National Center for University Entrance Examinations. Even then, such reforms would probably only be effective if they were applied to a single, centralized university entrance examination. As Watanabe (1996, p. 332) points out:
A large amount of time, money and energy is spent on entrance examinations every year at individual, school and national levels. In order to make the best use of such an investment, we need to be empirical. rational and well informed.
In this paper, I have listed some of the strategies available for promoting positive washback effects from the university entrance examinations. Such strategies could help improve the teaching and learning that is going on in Japan's junior and senior high school English language classrooms and at the same time help make the entrance examination process fairer and more relevant. I leave one question with the reader: Is Japan ready and willing to reform the current entrance examination system in order to foster positive washback effects that will help improve language education?

References

Bailey, K. M. (1996). Working for washback: A review of the washback concept in language testing. Language Testing, 13. 257-279.

Brown, J. D. (1987). False beginners and false starters: How can we identify them? JALT Journal, 11 (4), 9-11.

Brown, J. D. (1990). Where do tests fit into language programs? JALT Journal, 12 (1), 121-140.

Brown, J. D. (1993). Language test hysteria in Japan? The Language Teacher, 17 (12), 41-43.

Brown. J. D. (1995a). English language entrance examinations in Japan: Myths and facts. The Language Teacher, 19 (10), 21-26.

[ p. 6 ]


Brown J. D. (1995b). A gaijin teacher's guide to the vocabulary of entrance examinations. The Language Teacher, 19 (12), 25.

Brown, J. D. (1996a). English language entrance examinations in Japan: Problems and solutions. In G. van Troyer (Ed.) JALT '95: Curriculum and evaluation (pp. 272-283). Tokyo: Japan Association for Language Teaching.

Brown, J. D. (1996b). Japanese entrance exams: A measurement problem? The Daily Yomiuri (Educational Supplement). February 5, 1996. 15.


Brown, J. D. (1997). Do tests washback on the language classroom? TESOL ANZ Journal, 5, 63-80.

Brown, J. D. (1998). University entrance examinations and their effect on English language teaching in Japan. In J. Kalmy & M. James (Eds.), Perspectives on secondary EFL education: A publication in commemoration of the 30th anniversary of the Language Institute of Japan. (pp. 20-27). Odawara Japan: LIOJ.

Brown. J. D., & Christensen T. (1987). Interview: James D. Brown. The Language Teacher, 11 (7), 6-10.

Brown, J. D., & Gorsuch, G. (1995). An interview with J. D. Brown: Analyzing the value, meaning of entrance examinations. The Daily Yomiuri. (Educational Supplement). October, 30, 1995, 15.

Brown, J. D., & Kay, G. S. (1995). English language entrance examinations at Japanese universities: An interview with James Dean Brown. The Language Teacher, 19 (11), 7-11.

Brown, J. D., & Yamashita, S. O. (1995a). English language entrance examinations at Japanese universities: What do we know about them? JALT Journal, 17 (1), 7-30.

Brown, J. D., & Yamashita, S. O. (1995b). The authors respond to O'Sullivan's letter to JALT Journal: Out of criticism comes knowledge. JALT Journal, 17 (2), 257-260.

Brown, J. D., & Yamashita S. O. (1995c). English language entrance examinations at Japanese universities: 1993 and 1994. In J. D. Brown & S. O. Yamashita (Eds.) Language Testing in Japan (pp. 86-100). Tokyo: Japan Association for Language Teaching.

Gates, S. (1995). Exploiting washback from standardized tests. In J. D. Brown & S. O. Yamashita (Eds.), Language testing in Japan (pp. 101-106). Tokyo: Japan Association for Language Teaching.

Heyneman, S. P., & Ranson, A, W. (1990). Using examinations and testing to improve educational quality. Educational Policy, 4, 177-192.

Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press.

Kehaghan T. & Greaney, V. (1992). Using examinations to improve education: A study of fourteen African countries. Washington DC: The World Bank.

Messick, S. (1996). Validity and washback in language testing. Language Testing, 13,(3) 241-256.

O'Sullivan B. (1995). A reaction to Brown and Yamashita "English language entrance exams at Japanese universities: What do we know about them?" JALT Journal, 17 (2), 255-257.

Shohamy, E. (1992). Beyond performance testing: A diagnostic feedback testing model for assessing foreign language learning. Modern Language Journal, 76 (4), 513-521.

Shohamy, E. Donitsa-Schmidt S., & Ferman, I. (1996). Test impact revisited: Washback effect over time. Language Testing, 13 (3) 298-317.

[ p. 7 ]

Stapleton, P. (1996). A reaction to J. D. Brown's recent inquiry on the English entrance exam. The Language Teacher, 20 (3), 29-32.

Wall, D. (1996). Introducing new tests into traditional systems: Insights from general education and from innovation theory. Language Testing, 13,(3) 231-354.

Watanabe. Y. (1996). Does grammar translation come from the entrance examination? Preliminary findings from classroom-based research. Language Testing, 13 (3) 318-333.

Yoshida, K. (1996a). Language testing in Japan: A cultural problem? The Daily Yomiuri. (Educational Supplement). January 15. 1996, 15.

Yoshida, K. (1996b). Testing the bounds of culture. The Daily Yomiuri (Educational Supplement), February 12. 1996, 15.


NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join
STATISTICS CORNER ARTICLES:

#1   #2   #3   #4   #5   #6   #7   #8   #9   #10   #11   #12   #13   #14   #15   #16   #17   #18   #19   #20   #21   #22   #23   #24   #25   #26   #27   #28   #29   #30   #31   #32   #33   #34  
last Main Page next
HTML: http://jalt.org/test/bro_5.htm   /   PDF: http://jalt.org/test/PDF/Brown5.pdf

[ p. 7 ]