Theoretical frameworks of testing in SLA:
by Kayo Yoshida (Kansai Gaidai University)
|
|
This paper highlights some historical developments in language testing since the 1970's to the present and describes fundamental concepts and criteria which are thought to underpin good language testing. Discussing the importance of the interface and different perspectives between Language Testing (LT) and Second Language Acquisition (SLA) research, the author focuses on two studies by Skehan (1998) and Cohen (1998) which explore learners' test-taking process and strategies. This article suggests that it is essential to examine not only final linguistic products of tests but also the entire test taking process to make language tests more communicatively relevant. Keywords: language testing, test-taking strategies, test-taking process, historical developments, theoretical frameworks, SLA   |
Testing is considered a way to systematically measure a person's
ability or knowledge, and it is formalized as a set of techniques or procedures (Brown, 2001). Testing also plays an important
part in language learning and evaluation in classroom settings. In fact, a number of Second Language Acquisition (SLA) and
Language Testing (LT) researchers have discussed the roles of testing and its development in their fields. This paper provides
a historical overview of the development of those views over the last three decades. Some fundamental concepts and criteria for
good language testing will be also examined. Next, after pointing out some different viewpoints between SLA and LT, I will further
explore roles of language testing in SLA research by focusing on the knowledge and ability which underlie learners' performance.
Lastly, on the basis of two key SLA studies, I will argue that, in order to improve the quality of language testing and to provide
good opportunities for language learning, it is necessary to consider the test-taking process and the strategies required by the
communicative approaches which are currently recommended. In this regard Brindley (2001, p.139) states:
[ p. 2 ]
Many language tests and assessments used nowadays often contain tasks which resemble the kinds of language-use situations that test takers would encounter in using the language for communicative purposes in everyday life.
During the 1960s and 1970s, language testing techniques were
heavily influenced by structural linguistics (Chew, 2005). The analysis of language favoured by behaviourist approaches (e.g.
Skinner) led to discrete point testing, that is to say, tests were designed to assess learners' mastery of different areas of the
linguistic system in isolation [e.g. grammatical knowledge, vocabulary, pronunciation etc.] (reported by Bachman & Cohen 1998,
Brindley 2001). It was Chomsky (1965) who first rejected such approaches and proposed an underlying rule-based knowledge system.
From the early 1970s, however, communicative theories were widely adopted among linguistics and they began to focus on "communicative
proficiency rather than on mere mastery of structures" in language teaching (Richards 2001, p.153). This trend significantly
influenced the methods of language teaching and roles of language testing, although it is highly possible to assume that some
social changes induced new theories at first, and then the theories might be modified to support practice more closely. Hymes
took Chomsky's work further, but also reacted against some aspects of it. For Hymes (1972), the social context of language was
considered essential and appropriateness was viewed as important as grammatical correctness. Discrete-point teaching and testing
models were gradually replaced by models which aimed to integrate the various elements of language learning. A theory of
communicative competence was developed further by Canale and Swain (1980). They also raised two controversial issues related to
second language teaching and testing which will be explored later:
According to the new trends mentioned above by Richards (2001),
since the 1970s language testers have been seeking more pragmatic and integrative questions for assessment, such as cloze tests
and dictations. McNamara (2000, pp.14-15) points out the need by stating:
the necessity of assessing the practical language skills of foreign students led to a demand for language tests which involved an integrated performance on the part of the language user. The discrete point tradition of testing was seen as focusing too exclusively on knowledge of the formal linguistic system for its own sake rather than the way such knowledge is used to achieve communication.
For instance, Oller (1979) proposed the Unitary Competence
Hypothesis, which reflects the view that "performance on a whole range of tests depends on the same underlying capacity in the
learner - the ability to integrate grammatical, lexical, contextual, and pragmatic knowledge in test performance (McNamara 2000,
p.15)". This theory, however, is no longer accepted as a possible model of how language is processed.
In these days, with the widespread adoption of communicative
language teaching (CLT) principles, language tests tend to include more practical tasks predicting the real-world settings (see
Brindley, 2001). Although we still can encounter many discrete point questions, recent tests seem to have more diversity and
alternative evidence for assessment congruent with the communicative paradigm. For example oral examinations such as role-play
of speech acts, structured interview and information gap exercises contain tasks which reflect real-life demands are now common
in many language tests.
[ p. 3 ]
Although language testing has been influenced by social changes
such as those described above, there are certain fundamental aspects which remain widely accepted. According to Bell (1981,
pp.197-198), it is important to bear in mind what a test actually tests and how well it does so. Bell's view may be considered
together with Harris' analysis (1969, pp.21-22) which suggests that there are three key criteria to evaluate any 'good' test,
namely, reliability, validity and practicality.
It [a good test] must be reliable, that is, it must consistently give the same results under different conditions. It must also be valid, that is, it should really measure what it is supposed to measure. Testing experts also advise us to make sure a test is practical, that it is economical, easy to score, and easy to interpret.
Validity is particularly complex and may be broken down into a
number of facets, including content validity, criterion-related validity, construct validity, and face validity (see Hughes,
1989, pp.22-28). Bachman (2004, p.5) develops these views with six criteria for test usefulness: reliability, construct
validity, authenticity, interactiveness, impact and practicality. He examines the processes and strategies that test takers
use by investigating their individual responses and scores, in order to evaluate the test usefulness. This seems to be the
most comprehensive current model for designing, developing and using language tests.
Furthermore, there are other significant factors to take into
account in the process of language assessment. We might mention, in particular, the 'test rubrics' [e.g. the systematic
procedures used in a test] and the 'test-takers capacities' [i.e. their underlying knowledge and abilities]. These aspects
affect test performance in various ways (Chapelle and Brindley, 2002, pp.268-269). Thus, the features described above seem
to provide sufficient ground to produce and administer good language tests. However, we still need to examine what we exactly
intend to measure; competence or performance, and what the test scores exactly represent; learners' language ability or
background knowledge. These two sets of questions, as Canale and Swain proposed above, will be explored in this paper.
As previously stated, both LT and SLA researchers have influenced
one another and seem to have common interests and concerns. However, they differ in focus and perspective in some regards.
According to Bachman and Cohen (1998, pp.1-5), LT research has concentrated on language test performance and its products, that is,
the "results of acquisition". On the other hand, SLA research has been concerned with the "factors and processes" of
language acquisition and ability. A main focus has been on the development of interlanguage (see Selinker, 1972).
Especially, after the expansion of CLT models in the mid-1970s, SLA researchers began to emphasize language ability as
communication by including "non-language" factors such as personality and background knowledge. They began to look more toward
the cognitive information-processing involved in learning. For example, Bachman and Palmer (1996) examined the topical knowledge
of test-takers [knowledge of the world] and affective schemata [i.e. emotional memories influencing the way test-takers behave].
According to Schachter (1990 cited in Shohamy 1998, p.157), discourse knowledge is also important to involve both cultural
conventions and appropriate grammatical choices. Sasaki's (1993) study suggests that second language proficiency is related
to general cognitive abilities which include language aptitude and intelligence. Thus, a number of SLA researchers have focused
on learners' underlying competence which is likely to influence the performance/products of tests.
Let us now consider how learners' cognitive processes and strategies used during test taking related to the relationship
between competence and performance.
[ p. 4 ]
Here, we shall focus on the test-taking processes and
strategies used for oral communication in particular by drawing on these two recent studies:
A number of researchers have explored the relationship between
underlying competence and actual performance. Skehan (1998, pp.168-177) proposed that the testing approach of the "dual-mode"
model may be an effective way of explaining this difference especially in terms of assessing productive skills like oral proficiency.
The approach consists of abilities-driven scales to measure forms and rules [e.g. grammatical accuracy and complexity]
and processing-driven scales which mainly measure meaning and real-time processing [fluency]. Skehan stresses that the idea
of a parallel approach contributes to the implications for measuring competence-performance relationships, especially for
practical task-based testing. For example, spoken performance is likely to include areas such as grammar, vocabulary,
fluency, appropriateness, pronunciation, etc. Therefore, this approach offers a clear division when measuring competence -
or performance - based skills, even though the concepts themselves are not clear enough to separate.Test-taking strategies are frequently researched in SLA. It is
widely thought that language-use strategies include retrieval strategies, rehearsal strategies, cover strategies, and
communication strategies. Moreover, all four strategies are often involved in test taking (Cohen, 1998). Bachman and
Cohen (1998, p.14) mention that language-use strategies are "mental operations or processes" which learners consciously
select and apply in order to accomplish language tasks. Cohen stresses the importance of such strategies for producing
acceptable answers to questions and tasks before, during and after responding to them. Krashen (1982, p.180) presents
an example of a test-taking strategy to a obtain good score:
if the student were able to "cover" the problem with appropriate filler (just a moment . . . what I want to say is . .. how do you say . . . ?), he would not only not be penalized but would be graded up for having the ability to keep the conversation going and not lose the floor! Students would also be given credit for politeness and appropriateness, since a minimum amount of this knowledge is absolutely necessary for successful conversation.
This example shows that the student mainly employs
communication strategies by using fillers to gain time to think and appeal for aid from the interlocutor, so as to
compensate for missing knowledge as native speakers often do in conversation. Furthermore, Cohen's study focuses
on the process and variation of test-taking strategies. He examined 26 advanced English foreign language learners
and gave them six speech act situations (2 apologies, 2 complaints, 2 requests) in which they role-played with a native
speaker. The number of strategies they used for each task varied. For instance, one student said 'very sorry' in an
expression of apology, but reported thinking to herself afterwards that she could have said 'terribly sorry' [- Afterthought -].
This illustrates the existence of a gap between knowledge and performance. Conventional test scores/grades do not always
represent the ability of learners.
[ p. 5 ]
Another student in Cohen's study said, "When I start speaking English after not speaking it for a long time, my vocabulary
is weak and it is hard to retrieve words from memory" [- Retrieval process -]. In addition, some students planned their
answers in their L1, then translated them into a L2. However, others both planned and responded in L2. This suggests
that complicated processes are going on in learner's' brains, even when answers seem simple. It was also reported in Cohen's paper that, respondents devoted roughly twice as much attention to grammar than as they
did to pronunciation.
In SLA research, it is significant not only to examine learners'
final products - the results from testing but also to bear in mind the process how they think, choose and produce the appropriate
answers even in testing situation. I believe that this idea may also help improve language tests. Cohen (1998, p.37) emphasizes
that testing situations are good opportunities for language learning especially, "if the learner uses strategies for language
learning during the testing session". Krashen (1982, p.182) expresses a similar view by mentioning that even communicative
testing can contribute to the progress of grammatical accuracy:
if tests . . . encourage students to participate in conversation and develop the skills to manage conversations, they will contribute a great deal to the development of grammatical accuracy in the long run than any other kind of measure.
This article has briefly described a process of evolution in
language testing and discussed some aspects of good language testing and its main roles in the SLA area by focusing on learners'
test-taking processes and strategies. Considering significant changes which communicative theories brought in language education
fields, it is essential not only to consider appropriate testing to assess communicative skills such as oral proficiency but also
to bear in mind the cognitive processing strategies of the examinees. It is still difficult clearly to separate the concepts
between learner's competence and performance in language test situations. However, the "dual-mode [abilities- /processing-driven]
approach" proposed by Skehan (1998) may facilitate the measurement of competence-/performance based skills individually, and
may contribute to make measurement methods more effective. In terms of classroom teaching, there might be time constraints
to introduce ideal language tests congruent with this view and to assess learners properly due to limited teaching schedules.
Since this paper is merely based on theoretical perspective from the viewpoint of English language in SLA area, it seems possible
that the data may not be the same for other languages in different research fields. Despite these limitations, it is hoped to
get some idea from this study in designing and developing foreign language tests and then to obtain some evidence to satisfy
how they work well for the following studies. For further research, on the basis of Bechman's (2004) study, I shall explore
the importance of linking between theoretical features of language testing and practical procedures with statistical data included.
[ p. 6 ]
Author IndexTitle IndexDate Index
Background
Links
Network
Join
[ p. 7 ]