Book Reviews
Tips for Improving Testing and Grading.
by John C. Ory, & Katherine E. Ryan, (1993)
London: SAGE Publications. (ISBN 0-8039-4974-x)
Developing and Using Tests Effectively: A Guide for Faculty.
by Lucy C. Jacobs, & Clinton I. Chase, (1992)
San Francisco: Jossey-Bass Publishers. (ISBN 1-55542-481-3)
Language Testing and Evaluation.
Allison, Desmond (1999)
Singapore: Singapore University Press. (ISBN 9971-69-226-0)
These three books provide, in different ways, a general introduction to testing. However, their content and intended audiences vary.
Tips for Improving Testing and Grading is for American college professors and provides a basic introduction to the process
of designing, developing, administering and evaluating college tests. With a practical focus this book offers lots of examples of good
and bad test items and very little test theory.
Developing and Using Tests Effectively: A Guide for Faculty was also written for American college professors, and is similar
to the previous work. It does, however, cover theoretical issues more thoroughly and offers a more solid foundation.
Finally, Language Testing and Evaluation was written with ESOL MA students in mind and offers a solid theoretical background
to language testing, but includes no examples of good or bad test items to which the reader can relate the theory.
The first two books both use Bloom's (1956) taxonomy of cognitive levels alongside course content as the basis for outlining the specifications
for any particular test. Bloom identified six cognitive levels from simply memorising and being able to repeat information to analysis,
synthesis and evaluation. Both of these books sensibly point out that good tests need to cover all of these levels, and go on to show how
well designed "objective" items, especially MCQ, can be used to test all of Bloom's cognitive levels. However, it would be worth considering
how easily this system can be applied to testing language proficiency. Where as the content of a university course is usually very specific,
when we attempt to test language proficiency we are testing a content domain that is not as well defined. Consequently, appropriate sampling
of the content matter seems to be much more problematic.
Although Tips for Improving Testing and Grading is based on sound testing theory, that theory is only occasionally referred to.
One issue which seems to be oversimplified is the treatment of "objective tests", such as MCQ, T/F and so on. While the scoring of such
tests can be completely objective, the selection and design of the items can not be entirely objective. The subjective nature of this process
is not mentioned at all, which could leave readers with a distorted understanding of the issues surrounding the use of such tests.
While the content area of the first two books is almost entirely the same, Developing and Using Tests Effectively: A Guide for Faculty
deals with several issues which are not covered in Tips for Improving Testing and Grading at all. These issues are reliability
and validity, alternative assessment, computer assisted testing and item analysis. As reliability and validity are at the heart of all good
testing practices this is a valuable addition to this book. While alternative assessment methods are not widely used in most university
environments, it may be useful for readers to broaden their knowledge of these methods. It may also be interesting to consider issues
associated with performance testing in light of testing spoken language ability. With the increasing reliance on computers, it is
certain that some readers will want to know how computers can best be utilised to aid the testing process. This chapter deals mainly with
issues associated with the design and use of computer adaptive testing. While most readers are unlikely to use this information to develop
their own computer adaptive testing system, they may be interested to know that such procedures are possible in specialised testing contexts.
The chapter on item analysis deals only with the statistics which can be used to assist in the analysis of individual items and does not deal
with statistics which can be used to evaluate the performance of tests as a whole. Valuable as the information given is, it is a shame that
there is not also a discussion of how to evaluate tests as a whole.
It is worth considering how the first two books in this review could be used by EFL teachers. Although the testing principles are the same,
the examples do not relate to EFL and the issues surrounding the testing of language "proficiency" are not discussed. On the other hand,
while Language Testing and Evaluation deals with the various different theories of "language competence" (proficiency), it does not look in
such practical detail at how to construct test items. There are no clear guidelines to follow and there are no example items to highlight the
pitfalls an unsuspecting item writer might encounter.
[
p. 6
]
Language Testing and Evaluation is divided into two main sections. The first section deals with "contexts and perspectives"
and discusses theoretical issues such as: testing and evaluation in language teaching, evaluating language, describing and measuring language
ability, evaluating language curricula and testing in language classes. The second section deals with "issues and practices" and more practical
issues concerning language testing. However, these issues are discussed in a somewhat theoretical way. While the discussion is very interesting,
there does seem to be a fundamental mismatch between the theory of "language competence" discussed and described in Chapter 3 and the approach
to testing given in Chapters 8 & 9.
Chapter 3 of Allison's text deals with the development of theories about language competence. First, Oller's (1979) theory proposed a unitary
competence hypothesis; then various "four skills" (reading, writing, speaking and listening) approaches were suggested. This was followed by
Canale and Swain's (1980) ideas about communicative competence involving more than just knowledge of language itself. Finally, Bachman's
(1990) theory of communicative language ability incorporates many components such as context of situation, psychophysiological mechanisms,
strategic competence, knowledge structures and language competence. Therefore, to be fully communicative a user of the language must not only
know what all these components involve, but also be able to integrate and apply them appropriately.
Although Bachman argues that these aspects of language competence are interrelated, the book then goes on to present a four skills approach to
testing in Chapters 8 & 9. This approach segments language competence into reading, writing, speaking and listening (sometimes also grammar and
vocabulary). There appears to be very little explanation for this discrepancy and basically no attempt to provide an approach to language
testing based on Bachman's model. This leaves the reader wondering which of these two approaches the book is really advocating.
One of the features of this book is the use of "research tasks". Each major section has a task which is designed to promote deeper thinking
about the issues just discussed. Readers who complete all the suggested tasks will definitely deepen their understanding of language testing
issues. However, even MA students are unlikely to the time (or experience) to be able to complete all of these tasks as the author suggests.
Yet these tasks are a valuable addition to the book and the extent to which they achieve their full potential will depend largely on how much
time and effort the reader is prepared to invest in completing the tasks.
The only reason for EFL teachers in Asia to purchase Tips for Improving Testing and Grading is the way that it deals only with the essentials
of testing with brevity. Those wishing to explore testing issues in more depth would be well advised to read Developing and Using Tests
Effectively: A Guide for Faculty. Choosing between this text and Language Testing and Evaluation is more difficult.
While Chapter 3 was particularly interesting, I question whether either book provides the best theoretical introduction to language testing
theory and practice for EFL teachers.
Reviewed by Yvonne Ishida
References
Bloom, B. S. (Ed.) (1956) Taxonomy of educational objectives: The classification of educational goals: Handbook I, cognitive domain.
New York / Toronto: Longmans / Green.
Canale, M. and M. Swain. (1980) Theoretical bases of communicative approaches to Second language teaching and
testing. Applied Linguistics, 1, (p. 1 - 47)
Oller, J. W. Jr. (1979) Language Tests at School: A pragmatic approach. London: Longman.
Bachman, L. F. (1990) Fundamental Considerations in Language Testing. (Oxford Applied Linguistic Series). London: OUP.
PDF Version
