PDF Version
This book is a useful guide to test and survey design from a quantitative perspective for the benefit of those without a background in psychometrics, test design or psychology. The authors are both distinguished scholars at Cambridge University. John Rust is Director of the Cambridge University Psychometrics Centre and Susan Golombok is Director of the Centre for Family Research. The authors have collaborated on a number of different research inventories and tests such as the Measure of Gender Role Behavior in Pre-School Kids (Golombok & Rust, 1993) and the Measure of Marital State (Rust, et al., 1990).
The third edition, published 20 years after the first edition and a decade after the second, includes over 30 new pages of research and information about technological changes, as well as a number of fresh insights on standing controversies. An overview of the contents of this text follows, along with a note regarding its strengths and weaknesses.
The book starts off with a general introduction to the history of psychometrics. The discussion of the early history of testing in China is particularly intriguing, along with examples of how psychometric tests can be misunderstood and misused. The need for public accountability when using psychometric measures is emphasized. Issues regarding test construction and item analysis are covered next. Topics such as questionnaire design, norm and criterion-referenced tests, as well as classical test theory, latent trait models, and item response theory are deftly handled. There is also an accessible introduction to factor analysis covering correlation coefficients, factor identification and rotation, as well as how to avoid abusing factor analysis.
A thorough discussion of the use of psychometrics in education then follows. Topics such as the usefulness of school records and examination systems are underscored. After this, the authors aptly balance theoretical concerns such as theories of personality and personality assessment with practical concerns such as how to deal with uncooperative research participants. They also point out how to avoid Barnum effect problems, which occur when people accept broad-spectrum, vague statements as uniquely accurate descriptions of their condition. A pernicious example of the Barnum effect educators in Japan are more than likely familiar with is the Japanese popular theory of personality based on blood type. This has been used educationally, for example, to support kindergarten and athletic instruction according to a February 2009 Associated Press article.
[ p. 17 ]
An in depth discussion of personality assessment in the workplace is also provided in this text. The Big Five model of personality has become broadly accepted as a fundamental explanatory model of human personality. The Big Five, as investigated by researchers such as Digman (1990), Goldberg (1992), and Costa and McCrae (1992) is outlined. This model seeks to identify what factors are fundamental to the structure of human personality. The current version of this model includes five factors: openness to experience, conscientiousness, extraversion, agreeableness, and neuroticism. The first section of this book concludes with a discussion of ethics and psychometrics in the information technology age.
The final 22 pages of this volume provide a succinct, "how to", step-by-step guide on survey design. It explains important issues and points that need to be considered when designing surveys. Some of the advice and guidelines in this section will be useful for language teachers designing exams. For example, when determining the weighting of test sections, the blue print and matrix on pages 212-219 is useful. This will help test designers clarify issues such as whether to give more value and to create more test items for a given skill depending on what one teaches and wishes to assess. Also this section's rundown of the advantages and disadvantages of each type of test item format is helpful.
One attractive point about this book is that the authors are very careful to point out over and over the care that needs to be taken in constructing, applying, and interpreting psychometric data. Compelling examples of historical abuses of psychometric data and IQ testing are offered. As the authors point out, psychometrics touches nearly every aspect of modern life from medicine to education. Also, because factor analysis is such an important part of psychological research, those looking for a quick and concise introduction to it will find Chapter 7 particularly helpful.
One potential drawback of this book should be noted for readers familiar with item response theory. For example, the authors' treatment of item discrimination and respondent free measures regarding the Rasch model may be questionable, even contradictory. Linacre (2000) makes a strong rebuttal of the way that the Rasch model is presented in the previous edition (1999) of this work. Nothing in Rust and Golombok's current discussion appears to have changed or updated with respect to this issue. Rust and Golombok appear to envision a limited role for Rasch in psychometrics on page 66. However, on page 199 they also have some praise, or at least acknowledgment, of its utility.
This section consists of ten collections of activities designed to drive home the core concepts outlined previously. Ideally suited to small group work, the tasks can also be done in solo. In all but the first case, the connection between the task assignment and some aspect of language assessment is clear. Tasks include diverse activities such as crafting a school-wide assessment statement or reflecting on whether some tests could legitimately have gate keeping functions which do not necessarily foster learning. I particularly liked the final task: keeping a testing diary to note how your thoughts about the assessment process change over time.
As a general introduction to psychometric history, factor analysis and a "how to" for test and survey design, this book is worth having. Though relatively slim this volume provides an adequate treatment of issues involved in psychometric testing. Overall, as an entry-level introduction to psychometrics, this book deserves a place on the shelves of both testing neophytes and test designers. The theoretical and historical discussions provide a welcome review for experienced designers. The concise and clear guides for test design are a good entrance to the field for aspiring L2 researchers and test makers.
- Reviewed by Joseph E. Ring
[ p. 18 ]
Author IndexTitle IndexDate Index
Background
Links
Network
Join
[ p. 19 ]