Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 5 No. 3. Autumn 2001. (p. 7) [ISSN 1881-5537]

Book Review

Assessing language for specific purposes
by Dan Douglas (2000)ISBN: 0521585430, hard: 0521584957
Cambridge University Press (311 pages)

This text offers an overview of what language for specific purposes (LSP) tests are, how they are developed, how they are used, and how they are changing. It begins by defining LSP testing as a special form of communicative language testing. LSP tests such as the Japanese Language Test for Tour Guides differ from more general language tests such as the TOEFL® in two basic ways. First, the tasks in LSP tests reflect specific language use more than general tests do. Second, non-linguistic background knowledge plays a more significant role in LSP tests.

Countering claims by Widdowson, Davies, and others that LSP tests are not theoretically justified, the first four chapters of this book focus on a theoretical rationale for LSP testing. Many psycholinguistic and sociolinguistic concepts are expounded such as the relation between strategic competence, various domains of knowledge, internal discourse, and external context.

Chapter 5, arguably the heart of the book, investigates how to develop test tasks indicative of specialized target language use. It provides detailed guidelines for constructing LSP tests based on a model of test development expounded by Bachman and Palmer (1996). Frequent problems in constructing LSP tests are narrated as well. One frequent problem is that many LSP test developers do not have a detailed knowledge of the specialized subject they are writing a test for. To compensate for their lack of background knowledge, the opinions of expert informants are often indispensable. Other ways of analyzing language use in a specific domain by using context-based research (Douglas & Selinker, 1994), and grounded ethnography (Frankel & Beckman, 1982) are considered. In both cases, considerable time is needed to do an adequate task of assessing how language is used in a specific target situation and translating that knowledge into test items.

Another problem in some LSP tests is that raters lack expertise in the specialized domain of knowledge being evaluated. Depending on the nature of the response examinees are expected to give and the testing environment, this can be potentially a serious threat to validity.
"Today LSP test tasks frequently have a high degree of situational authenticity (reflecting actual target language use characteristics), but a low degree of interactional authenticity (failing to engage examinees in communicatively purposeful activities.)"

Considering the entire test development process, Douglas adds, ". . . the most difficult aspect of producing test specifications is making the leap from the analysis of the target language use tasks to the specifications of test tasks" (p. 113). The author notes there are no idiot-proof guidelines when it comes to creating test items, but the modified framework of Bachman and Palmer's (1996) model for test development discussed in this book makes the task easier.

Throughout the book, Douglas reiterates the need to apply the characteristics of good testing practise which, among other things, involve thorough piloting as well as considering a number of validity factors. "Validation is not a once-and-for-all event," the author states, "but rather a dynamic process in which many different types of evidence are gathered and presented in much the same way as a mosaic is constructed. . . " (p. 258). By using a variety of formal statistical measures as well as a qualitative feedback from a wide variety of sources, we can begin to get a better idea of what a test is actually doing.

Chapters 6 and 7 of the book examine several specific LSP tests. Characteristics of exams such as the Occupational English Test, IELTS, UETESOL are analyzed according to a framework established by Wesche (1983) in order to see how the principles for test development highlighted earlier in the book are carried out in actual practise. Douglas gauges the strengths and weaknesses of several LSP tests, noting how LSP tests often have a wide variety of specificity, broad range of input and response types, but a narrow range of assessment criteria. In most LSP tests, assessment criteria have fairly narrow linguistic focus. Douglas concedes, ". . . the development of communicative, specific purpose assessment criteria is emerging as one of the most vexing and problematic aspects of LSP testing. . . " (p. 245).

Assessing Language for Specific Purposes concludes with an overview of some still unresolved issues in LSP testing. One issue concerns the nature of input itself. Douglas expresses concern about obtaining genuine input in testing, remarking that:
the very nature of the LSP testing enterprise means that there will always and inevitably be a reduction in the dynamic interplay between the test taker and the characteristics of the test task, because a test is, by definition, a controlled and contrived environment. (p. 278)

Another issue of concern is the extent to which test tasks can engage communicative abilities. Too often, test tasks involve minimal negotiation for meaning or creation of discourse. Today LSP test tasks frequently have a high degree of situational authenticity (reflecting actual target language use characteristics), but a low degree of interactional authenticity (failing to engage examinees in communicatively purposeful activities.) One reason for this is more and more people are relying on computers to correct tests, and computer scoring typically utilizes a format such as multiple choice. Many test development issues ultimately boil down to issues of time and money and it is often cheaper to have computers correct tests than highly trained raters.

Noting the tendency of many institutions to favor general language tests over LSP tests and the ongoing criticisms of LSP tests themselves as being atheoretical, unnecessary, or unreliable, Douglas concludes this work by underscoring the intrinsic value of LSP tests and the need to ascertain some key psycholinguistic issues more clearly. "If we want to know how well individuals can use language in specific contexts of use," he states "we will require a measure that takes into account both their language knowledge and their background knowledge . . ." (p. 282).

This volume is of particular value to test developers and LSP teachers. Since this book attempts to grapple with so many diverse and often theoretical issues, it is a daunting task to read from cover-to-cover. However, its comprehensive index and well-researched annotations make this work a valuable reference for all those interested in LSP testing.

Reviewed by Tim Newfields

Works Cited

Bachman, L. & Palmer, A. (1996). Language testing in practice. Oxford: Oxford University Press.

Davies, A. (1984). ESL expectations in examining: the problem of English as a foreign language and English as a mother tongue. Language Testing. 1, 82 - 98.

Douglas, D. & Selinker L. (1994). Research methodology in context-based second language research. In E. Tarone, S. Gass, & E. Cohen (eds.), Methodologies for eliciting and analyzing language in context. Northvale, NJ: Erlbaum, pp. 119 - 131.

Frankel, R. & Beckman, H. (1982). IMPACT: an interaction based method for preserving and analyzing clinical transactions. In L. Pettigrew (ed.), Explorations in provider and patient transactions. Memphis, TN: Humana.

Wesche, M. (1983). Communicative testing in a second language. Modern Language Journal, 67 (1), 41- 55.

Widdowson, H. (2001). Communicative language testing: The art of the possible. In C. Elder, A. Brown, et al. (eds.), Experimenting with uncertainty: Essays in honour of Alan Davies. (pp. 12-21). Cambridge: Cambridge University Press.

[ p. 7 ]

NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join
last Main Page next
Copyright (c) 2001 by Tim Newfields. All rights reserved.
www.jalt.org/test/new_1.htm