Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 10 No. 1. Mar. 2006. (p. 21 - 26) [ISSN 1881-5537]

Statistics Corner
Questions and answers about language testing statistics:

Resources available in language testing

 Photo of JD Brown, c. 2000
James Dean Brown
University of Hawai'i at Manoa


* QUESTION: I'm new to language testing but have taken one language testing course and am very interested in learning more. Can you tell me where I can get more information so I can keep on learning about language testing?

* ANSWER: In recent years, more and more people seem to share your interest in language testing, especially in Japan. I would suggest that you start with the following: (a) check out several Internet websites that I will give below, (b), subscribe to one or more language testing journals, (c) join a language testing organization, and (d) read some of the many books that have recently been published.

Check Out Several Internet Websites

A number of Internet websites specialize in language testing issues, but the single best source of information is the "Resources in Language Testing Page" maintained by Glenn Fulcher at http://www.le.ac.uk/education/testing/ltr.html. Fulcher's webpage includes links, reviews, articles, video FAQS, and a searchable database. The links are particularly interesting because they will take you to other websites that he labels as private/commercial websites; personal pages; government websites; ERIC; associations, centers, and councils; universities; ethics and fairness; statistics; tests online; computer based testing; search for a specific test; conferences; awards; ILTA; EALTA; LTRC; and LTEST-L. You may also find "The Language Tester's Guide to Cyberspace" (also from Fulcher) useful at http://www.le.ac.uk/education/testing/ltrfile/cybertxt.html. It provides an excellent general guide to websites listed above.

Bob Godwin-Jones provides other websites on language testing (some of which are unfortunately out of date) in his 2001 article entitled "Emerging technologies: Language testing tools and technologies" [Language Learning & Technology, 5(2), 8-12] available at http://llt.msu.edu/vol5num2/emerging/default.html. His article covers the following topics with many links available within each: computerized testing; internet applications; authoring tools; outlook; and a resource list (including web-based testing resources; organizations and institutions; language tests; sample on-line practice tests; language placement tests on-line; test makers, tools, and templates).

A number of other generic testing and evaluation clearinghouses can also be found on the internet. Check out the following:

Subscribe to One or More Language Testing Journals

At the moment there are two primary language testing journals that serve as the gold standard for research in our sub-field. You should definitely consider subscribing to the following two journals at the following websites:

[ p. 21 ]

And, don't forget to skim through them when they start to arrive.

Clearly, if you are reading this article, you already know about the JALT Testing and Evaluation SIG newsletter called the SHIKEN, but you may not know that it is regularly available as a member of the JALT Testing and Evaluation SIG or that, after publication, the individual articles are available at http://jalt.org/test/pub.htm. Another newsletter that may be interesting to language testers is the ILTA Newsletter at http://www.le.ac.uk/education/testing/ltr.html.

Naturally, you should also keep and eye out for articles on language testing in other mainstream second language journals like: Applied Linguistics, JALT Journal, The Language Teacher, Language Learning, Language Teaching, Language Teaching Research, Modern Language Journal, RELC Journal, Studies in Second Language Acquisition (SSLA), TESOL Quarterly, and so forth.

If you find yourself getting pathologically serious about language testing, you will probably want to become conversant with one or more of the following measurement journals in education and psychology:

Join a Language Testing Organization

One way to keep in touch with other language testers is to join a language testing organization. Some readers may be surprised to learn that there are a number of organizations that promote language testing. Premier among these is the International Language Testing Association (ILTA) http://www.iltaonline.com/. However, other regional organizations may be of particular interest to those language testers who happen to live within those particular regions:

[ p. 22 ]

One of the primary benefits of joining these organizations is that they often sponsor conferences and workshops. Such events are a useful way to learn about language testing, but they can also help you begin establishing networks of friends and acquaintances in the field. Check the websites above for more on the various conferences and workshops that are going on near you.

Read Some of the Recently Published Books

The following is probably not a complete list of the language testing books that have been published since 1995. However, they are the recent language testing books that I have on my shelf or have on order, and they will serve as a good starting point for reading up on the field. New language testers should scan through the list for titles or authors that interest them, order the book from the publisher or on Amazon.com, and start reading. More established language testers may want to scan through the list to see if there is anything they have missed. So here is my (fairly complete) list of books on language testing published since 1995:

Alderson, C. J. (2000). Assessing reading. Cambridge: Cambridge University (ISBN: 0521599997).

Alderson, C. J., Clapham, C., & Wall, D. (1995). Language test construction and evaluation. Cambridge: Cambridge University (ISBN: 0521478294).

Allison, D. (1999). Language testing and evaluation: An introductory course. Singapore: Singapore University and World Scientific (ISBN: 9971692260).

Bachman, L. F. (2004). Statistical analyses for language assessment. Cambridge: Cambridge University (ISBN: 0521003288).

Bachman, L. F., & Cohen, A. D. (Eds.). (1998). Interfaces between second language acquisition and language testing research. Cambridge: Cambridge University (ISBN: 0521649633).

Bachman, L. F., & Kunnan, A. J. (2005). Statistical analyses for language assessment workbook and CD ROM. Cambridge: Cambridge University (ISBN: 0521609062).

Bachman, L. F., & Palmer, A. S. (1996). Language testing in practice: Designing and developing useful language tests. Oxford: Oxford University (ISBN: 0194371484).

Bachman, L. F., Davidson, F., Ryan, K., Inn-Chull, C. (1995). Studies in language testing 1: An investigation into the comparability of two tests of English as a foreign language. Cambridge: Cambridge University (ISBN: 0521484677).

Bailey, K. (1997). Learning about language assessment: Dilemmas, decisions, and directions. Washington, DC: International Thomson (ISBN: 0838466885).

Banerjee, J., Clapham, C., Clapham, P., & Wall, D. (1999). ILTA language testing bibliography 1990-1999. Lancaster, UK: Lancaster University (ISBN: 1862200734) (the same material is available at http://www.iltaonline.com/ILTA_pubs.htm).

Barnwell, D. P. (1996). A history of foreign language testing in the United States from its beginning to the present. Tempe, AZ: Bilingual Review. (ISBN: 0927534592).

Blue, G. M., Milton, J., Saville, J. (Eds.). (2000). Language testing and evaluation 1: Assessing English for academic purposes. Frankfurt: Peter Lang (ISBN: 0820453161).

Brown, H. D. (2003). Language assessment: Principles and classroom practices. New York: Pearson Longman ESL (ISBN: 0130988340).

Brown, J. D. (1996). Testing in language programs. Upper Saddle River, NJ: Prentice Hall (ISBN: 0131241575).

Brown, J. D. (2005). Testing in language programs: A comprehensive guide to English language assessment (New ed.). New York: McGraw-Hill College (ISBN: 0072948361).

Brown, J. D. (Ed.). (1998). New ways of classroom assessment. Alexandria, VA: TESOL (ISBN: 0939791722).

Brown, J. D., & Hudson, T. (2002). Criterion-referenced language testing. Cambridge: Cambridge University (ISBN: 0521000831).

Brown, J. D., & Yamashita, S. O. (Eds.). (1995). Language testing in Japan. Tokyo: Japan Association for Language Teaching (ISBN: 4990037006).

Brown, J.D. (translated into Japanese by M. Wada). (1999). Gengo tesuto no kisochishi. [Basic knowledge of language testing]. Tokyo: Taishukan Shoten (ISBN 4469212261).

Brown, J.D., Hudson, T., Norris, J.M., & Bonk, W. (2002). Investigating second language performance assessments. Honolulu, HI: University of Hawaii (ISBN 0824826337).

[ p. 23 ]


Buck, G. (2001). Assessing listening. Cambridge: Cambridge University (ISBN: 0521666619).

Chalhoub-Deville, M. (2000). Studies in language testing 10: Issues in computer-adaptive testing of reading proficiency. Cambridge: Cambridge University (ISBN: 0521653002).

Chapelle, C., & Douglas, D. (2006). Assessing language through computer technology. Cambridge: Cambridge University (ISBN: 0521549493).

Cheng, L. (2005). Studies in language testing 21: Changing language teaching through language testing: A washback study. Cambridge: Cambridge University (ISBN: 0521544734).

Cheng, L., & Watanabe, Y. (Eds.). (2004). Washback in language testing: Research contexts and methods. Mahwah, NJ: Lawrence Erlbaum Associates (ISBN: 0805839879).

Clapham, C. (1996). Studies in language testing 4: The development of IELTS: A study of the effect of background on reading comprehension. Cambridge: Cambridge University (ISBN: 0521567084).

Clapham, C., & Corson, D. (Eds.). (1997). Encyclopedia of language and education: Volume 7, Language testing and assessment. Dordrecht, NL: Kluwer Academic (ISBN: 0792349342).

Cohen, A. D. (1994). Assessing language ability in the classroom (2nd ed.). Boston: Heinle & Heinle (ISBN: 0838442625).

Coombe, C. A., & Hubley, N. J. (2003). Assessment practices. Alexandria, VA: TESOL (ISBN: 1931185077)

Cumming, A., & Berwick, R. (1996). Validation in language testing. Clevedon, UK: Multilingual Matters (ISBN: 1853592951).

Davidson, F., & Lynch, B. K. (2002). Testcraft: A teacher's guide to writing and using language test specifications. New Haven, CT: Yale University (ISBN: 0300090064).

Davies, A. (forthcoming). Studies in language testing 23: Testing English for academic purposes, 1950-2005. Cambridge: Cambridge University (ISBN: 0521834732).

Davies, A., Brown, A., Elder, C., Hill, K., Lumley, T., & McNamara, T. (1999). Studies in language testing 7: Dictionary of language testing. Cambridge: Cambridge University (ISBN: 0521658764).

Douglas, D. (2000). Assessing language for specific purposes. Cambridge: Cambridge University (ISBN: 0521585430).

Ekbatani, G., & Pierson, H. (Eds.) (2000). Learner-directed assessment in ESL. Mahwah, NJ: Lawrence Erlbaum Associates (ISBN: 0805830677).

Elder, C., Brown, A., Grove, E., Hill, K., Iwashita, N., Lumley, T., McNamara, T., & O'Loughlin, K. (Eds.) (2001). Studies in language testing 11: Experimenting with uncertainty: Essays in honour of Alan Davies. Cambridge: Cambridge University (ISBN: 0521772560).

Fulcher, G. (2003). Testing second language speaking. Cambridge: Cambridge University (ISBN: 0582472709).

Genessee, F., & Upshur, J. A. (1996). Classroom-based evaluation in second language education. Cambridge: Cambridge University (ISBN: 0521566819).

Gottlieb, M. (2006). Assessing English language learners: Bridges from language proficiency to academic achievement. Thousand Oaks, CA: Corwin (ISBN: 0761988882).

Green, A. (1998). Studies in language testing 5: Using verbal protocols in language testing research: A handbook. Cambridge: Cambridge University (ISBN: 0521584132).

Hasselgreen, A. (2005). Studies in language testing 20: Testing the spoken English of young Norwegians: A study of testing validity and the role of 'smallwords' in contributing to pupils' fluency. Cambridge: Cambridge University. (ISBN: 0521544726).

Hawkey, R. (2005). Studies in language testing 16: A modular approach to testing English language skills: The development of the Certificates in English Language Skills (CELS) examinations. Cambridge: Cambridge University (ISBN: 0521013321).

Hawkey, R. (forthcoming). Studies in language testing 24: Impact theory and practice: Studies of the IELTS test and Progetto Lingue 2000. Cambridge: Cambridge University.

Hudson, T., & Brown, J. D. (Eds.). (2001). A focus on language test development: Expanding the language proficiency construct across a variety of tests. Honolulu, HI: University of Hawaii (ISBN 0824823516).

Hudson, T., Detmer, E., & Brown, J. D. (1995). Developing prototypic measures of cross-cultural pragmatics. Honolulu, HI: University of Hawaii (ISBN 082481763X).

Hughes, A. (2002). Testing for language teachers (revised). Cambridge: Cambridge University (ISBN: 0521484952).

Kunnan, A. J. (1996). Studies in language testing 2: Test taker characteristics and performance: A structural modeling approach. Cambridge: Cambridge University (ISBN: 0521484669).

[ p. 24 ]


Kunnan, A. J. (2000). Studies in language testing 9: Fairness and validation in language assessment: Selected papers from the 19th Language Testing Research Colloquium, Orlando, Florida. Cambridge: Cambridge University (ISBN: 0521658748).

Kunnan, A. J. (Ed.). (1998). Validation in language assessment. Mahwah, NJ: Lawrence Erlbaum Associates (ISBN: 0805827536).

Lazaraton, A. (2002). Studies in language testing 14: A qualitative approach to the validation of oral language tests. Cambridge: Cambridge University (ISBN: 052180227X).

Luoma, S. (2004). Assessing speaking. Cambridge: Cambridge University (ISBN: 0521804876).

McKay, P. (2006). Assessing young language learners. Cambridge: Cambridge University (ISBN: 0521601231).

McNamara, T. (1996). Measuring second language performance. London: Longman (ISBN: 0582089077).

McNamara, T. (2000). Language testing. Oxford: Oxford University (ISBN: 0194372227).

Milanovic, M., & Saville, N. (Eds.) (1995). Studies in language testing 3: Performance testing, cognition and assessment: Selected papers from the 15th Language Testing Research Colloquium, Cambridge and Arnhem. Cambridge: Cambridge University (ISBN: 0521484465-0).

Milanovic, M., & Weir, C. J. (Eds.) (2004). Studies in language testing 18: European language testing in a global context: Proceedings of the ALTE Barcelona Conference July 2001. Cambridge: Cambridge University (ISBN: 052182897X).

Norris, J. M., Brown, J. D., Hudson, T., & Yoshioka, J. (1998). Designing second language performance assessments. Honolulu, HI: University of Hawaii (ISBN: 0824821092).

O'Loughlin, K. J. (2001). Studies in language testing 13: The equivalence of direct and semi-direct speaking tests. Cambridge: Cambridge University (ISBN: 052166098X).

O'Malley, J. M. (1996). Authentic assessment for English language learners. Boston: Addison Wesley (ISBN: 0201591510).

O'Sullivan, B. (forthcoming). Studies in language testing 17: Issues in testing business English: The revision of the Cambridge Business English Certificates. Cambridge: Cambridge University (ISBN: 0521013305).

Purpura, J. E. (1999). Studies in language testing 8: Learner strategy use and performance on language tests: A structural equation modeling approach. Cambridge: Cambridge University (ISBN: 0521658756).

Purpura, J. E. (2004). Assessing grammar. Cambridge: Cambridge University (ISBN: 052100344X).

Read, J. (2000). Assessing vocabulary. Cambridge: Cambridge University (ISBN: 0521627419).

Röver, C. (2005). Language testing and evaluation 2: Testing ESL pragmatics: Development and validation of a web-based assessment battery. Frankfurt: Peter Lang (ISBN: 082047343X).

Sigott, G. (2004). Language testing and evaluation 1: Towards identifying the C-test construct. Frankfurt: Peter Lang (ISBN: 0820465585).

Spolsky, B. (1995). Measured words. Oxford: Oxford University (ISBN: 0194372014).

Taylor, L. & Falvey, P. (Eds.) (forthcoming). Studies in language testing 19: IELTS collected papers: Research in speaking and writing assessment. Cambridge: Cambridge University.

Teachers of English to Speakers of Other Languages (TESOL). (2001). Scenarios for ESL Standards-Based Assessment. Alexandria, VA: TESOL (ISBN: 0939791900).

University of Cambridge Local Examination Syndicate (UCLES). (1999). Studies in language testing 6: Multilingual glossary of language testing terms. Cambridge: Cambridge University (ISBN: 0521658772).

University of Cambridge Local Examination Syndicate (UCLES). (1999). Studies in language testing 6: Multilingual glossary of language testing terms (Audio CD). Cambridge: Cambridge University (ISBN: 0521658241).

Wall, D. (2005). Studies in language testing 22: The impact of high-stakes testing on classroom teaching: A case study using insights from testing and innovation theory. Cambridge: Cambridge University (ISBN: 0521542499).

Weigle, S. C. (2002). Assessing writing. Cambridge: Cambridge University (ISBN: 0521784468).

Weir, C. J. & Milanovic, M. (2003). Studies in language testing 15: Continuity and innovation: Revising the Cambridge Proficiency in English Examination 1913-2002. Cambridge: Cambridge University (ISBN: 0521813506).

Weir, C., Huizhong, Y., & Yan, J. (2000). Studies in language testing 12: An empirical investigation of the componentiality of L2 reading in English for academic purposes. Cambridge: Cambridge University (ISBN: 0521653819).

Yamashita, S. O. (1996). Six measures of JSL pragmatics. Honolulu, Hawaii: University of Hawaii. (ISBN: 0824819144).

[ p. 25 ]

Conclusion

As you are checking out some websites, subscribing to a couple of language testing journals, joining a language testing organization (and attending their conferences and workshops), and reading a half dozen books on language testing, you might also consider doing some actual nuts and bolts language testing. Such nuts and bolts involve getting your hands dirty by doing some actual language test development, which in turn involves writing test items, administering the items, item analyzing the results, revising the test on the basis of those results, validating the test, and doing research based on the test. Since doing language testing is the fun part, I recommend getting started with that as soon as possible. Once you take this hands on step, you'll truly be hooked. Enjoy!

Where to Submit Questions:

Please submit questions for this column to the following e-mail or snail-mail addresses:

brownj@hawaii.edu

JD Brown, Department of Second Language Studies
University of Hawai'i at Manoa
1890 East-West Road, Honolulu, HI 96822 USA


NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join

STATISTICS CORNER ARTICLES:
#1   #2   #3   #4   #5   #6   #7   #8   #9   #10   #11   #12   #13   #14   #15   #16   #17   #18   #19   #20   #21   #22   #23   #24   #25   #26   #27   #28   #29   #30   #31   #32   #33   #34  
last Main Page next
http://jalt.org/test/bro_23.htm

[ p. 26 ]