Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 11 No. 2 Sep. 2007. (p. 2 - 5) [ISSN 1881-5537]
PDF PDF Version

MEXT's 2003 Action Plan:
Does it encourage performance assessment?

Kristie Sage   (Komazawa University)


Abstract

This qualitative research paper investigates some aspects of Japan's Ministry of Education, Culture, Sports, Science and Technology Action Plan to cultivate Japanese students' English aptitude. While the main purpose of the Action Plan appears to be to promoting communicative English ability, problematic issues exist. One problem highlighted in this paper is that the four language skills do not represent a good construct of language ability. Secondly, the English section of the National Centre Test used for many university entrance exams and endorsed by MEXT's Action Plan, lacks validity. This article concludes by introducing models of communicative ability that inform performance assessment.
Keywords: MEXT's Action Plan, educational planning, communicative English, four language skills, NCT, performance assessment

Japan's Ministry of Education, Culture, Sports, Science and Technology (MEXT) officially began to support a communicative language orientation in the mid-1980s for junior and senior high schools in the area of practical communication skills (O'Donnell, 2005). Further, curricular reform reports from two decades ago more or less emphasized communicative competence through the Courses of Study (Koike & Tanaka, 1995; Monbusho, 1999; 2003; Tanabe, 2004 as cited in O'Donnell, 2005; MEXT, 2003, p.4). Most recently, MEXT's Action Plan was established in 2003 to promote the communicative English ability of junior and senior high school, as well as university, students to make them more "internationalized". This Action Plan incorporates many of the principles and practices of the Courses of Study (MEXT, 2003). Its main goal is avowedly to "cultivate Japanese with English abilities" (MEXT, 2003). However, as O'Donnell (2005) cites, concerns have been raised by the former Education Ministry (1999, 2003) regarding the operationalisation of Communicative Language Teaching (CLT) reforms.
Even though MEXT's Action Plan appears to focus on ways to foster communicative ability, many obstacles currently exist; namely, the reputed complex nature of testing communicative ability of a foreign language. This paper will discuss that issue from the perspective of performance assessment which by definition, ". . . in language tests, [is] a class of test in which assessment is carried out in a context where the candidate is involved in an act of communication" (McNamara, 2000, p. 135). Prompting the research question: 

	"Does MEXT's 2003 Action Plan to cultivate English ability 
	encourage performance assessment among Japanese students?"

To qualitatively investigate this question, three sub-topics will be explored. Firstly, second language communicative ability and performance will be examined. Next, the trends of language testing, especially communicative ability models and the role of performance assessment are considered. Finally the MEXT Action Plan is discussed in terms of these themes, simultaneously with the four skills as promoted by the English sections of the National Centre Test (NCT).

[ p. 2 ]

Second language communicative ability and performance


The title of Chapter 3 of McNamara's 1996 book on second language performance assessment is intriguing. Entitled Modeling performance: Opening Pandora's box, it introduces the Pandora's box metaphor, which was originally coined by Canale and Swain (1980). McNamara suggests the much debated role of performance in second language ability can be likened to a Pandora's box. Another main point is his discussion of the role played by performance in second language ability – drawing on work from other influential writers in this field, including Bachman, Palmer, Canale and Swain. The basis of this chapter stems from the nature of performance being summarized to consist of three main facets: (1) the factors which constitute language knowledge, including organizational knowledge (grammatical and textual) and pragmatic knowledge (lexical, functional and sociolinguistic) (Celce-Murcia et. al. 1995), (2) the non-language factors which contribute in part to "an individual's ability to perform communicative tasks involving language", termed by Hymes in 1972 "ability for use" (McNamara, 1996, p.48). That is, the cognitive and affective variables, for example the candidate, rater and score behavior relationship (McNamara, 1996, p. 54), gender in oral interviews (Porter, 1991 cited by McNamara, 1996, p. 54), group or pairing effects in oral tasks (Iwashita, 1993, cited by McNamara, 1996, p. 54), and (3) "actual real-time instances of language use" in terms of (1) and (2) (McNamara, 1996, p. 48). As a case in point, a Japanese high school EFL learner might have some grammatical and lexical knowledge - facet (1) - yet may be hindered from the effect of variables - facet (2) - such as an incompatible pairing with another test candidate. In short, McNamara reasons that the fundamental nature of performance testing is practical and ". . . could very nicely get along without the assistance of appropriate linguistic theory" (p. 49).

From discrete point to models of communicative ability

McNamara (2000, p. 65) remarks on the trend to move away from multiple choice items (typical of language tests used by for example the NCT) to assessing integrated performances, a characteristic of communicative approaches of language teaching. In saying this, it is important to note that this trend has not occurred overnight. Rather, as McNamara (2000, pp. 13-22) documents, in brief, the trend has moved from discrete point, to integrative and pragmatic, to communicative language tests, and currently, to models of communicative ability, each superseding the earlier. (For future research it would be beneficial to ascertain where those tests that the MEXT Action Plan endorses, such as the TOEFL®, TOEIC®, STEP-Eiken, University of Cambridge ESOL examinations, and NCT are placed on the performance assessment continuum.) McNamara (1996, p. 85) states, "A weakness of current models is that they focus too much on the individual candidate rather than the candidate in interaction. Given the interactional nature of performance assessment we should be looking more to those in our field who are studying talk in interaction."

One problem with MEXT's 2003 Action Plan:
The non-interactive and non-performance related goals of the NCT

"to cultivate Japanese with English abilities, teaching and the subsequent testing of communicative ability cannot be overlooked as it currently is."


It could be said that in order for MEXT's Action Plan to cultivate Japanese with English abilities, teaching and the subsequent testing of communicative ability cannot be overlooked as it currently is. Let us consider the Action Plan's goals which relate to the NCT, as this test is taken by about half a million high school students in Japan annually (Suzuki, 2006, n.p.).
NCT attempted to fulfill MEXT Action Plan goals in the academic year 2005/2006 when a separate listening section was included. Since performance is a requirement of communicative language testing (McNamara, 2000, p. 17), and by definition performance tests generally require extended samples of speaking or writing to be elicited from the test taker in contexts that simulate tasks found in the real world (McNamara, 2000, p. 6), the inclusion of a listening section warrants critique. Its discrete format is obsolete and it fails to account for any type of interaction, crucial for performance assessment of communicative language ability.
The following selected statements can be found in the English translation of MEXT's Action Plan. Section 4 is entitled Improvement in the evaluation system for selecting school and university applicants to support the aforementioned. It can be seen from these goals that the listening test features prominently –

[ p. 3 ]

Goals: 1) Communication abilities, including listening and speaking will be appropriately evaluated. 2) Utilization of listening or external proficiency examinations will be encouraged for entrance examinations of universities and high schools. 1. Introduction of a listening test in the University Center examination 2. Promotion of improvements in the selection of applicants at each university É in particular, from a perspective of emphasizing communication abilities, selection methods that appropriately evaluate communication abilities will be promoted in the approach of each university through the use of such means as listening tests. 3. Promotion of the use of results of external proficiency examinations in the entrance examinations of universities and high schools.- 2003 MEXT Action Plan, Section 4, p. 13 [Bold type added by the author for emphasis]

Another problem with MEXT's Action Plan:
not correctly assessing communicative ability

By attempting to assess communicative ability without paying adequate attention to the measurement of performance, MEXT's Action Plan is off target. Instead of assessing communicative ability, the English listening skill found in the NCT test attempts to measure listening receptively and in virtual isolation, which is inappropriately considered an effective means to measure students' communicative language ability. (Ways that the current version of the NCT listening test fail to do this are summarized in Sage and Tanaka, 2006).
What appears to be the case is that MEXT is not as informed as they should be of what performance testing involves. Indeed, McNamara (2000, pp. 19-21) acknowledges the complexity of the factors which constitute performance and standardizing assessment. Yet, he reiterates the need for test performance to be of social character; in other words, "instead of focusing on the candidate in isolation, the candidate's performance needs to be seen and evaluated as part of a joint construction by a number of participants, including interlocutors, test designers and raters" (McNamara, 2000, p.21). Clearly the NCT does not facilitate this type of communicative ability assessment.

Discussion: performance and communicative competence

McNamara's (1996) chapter on modelling performance provides a succinct historical overview of the progressions made in testing models related to performance. Further, he discusses at great lengths the diversity of opinions regarding performance assessment among prominent academics such as Hymes, Canale, Swain, Bachman, and Palmer. Indeed, this chapter serves to show just how challenging testing communicative ability is. Conversely, the current trend of using models of communicative ability does show consensus among these researchers in terms of ". . . [the] continuing theoretical engagement with the idea of communicative competence and its implications for the performance requirement of communicative language testing (McNamara, 2000, p. 17). Agreeing that communicative competence encompasses the social roles that test candidates will assume in real world settings; tests should be able to do this by measuring grammatical, sociolinguistic, strategic, discourse and actional sub-competencies (McNamara, 2000, pp. 17-18; Celce-Murcia, Dornyei, & Thurrel, 1995).

[ p. 4 ]

Conclusions

This article has argued that performance assessment is an inextricable aspect of communicative language testing. It has also criticised the adoption of the listening test in the Japanese National Center Exam as an unsuccessful attempt to realize MEXT's 2003 Action Plan. In light of current communicative language testing theory, this paper has also suggested that MEXT must consider moving away from promoting norm-reference test formats. In doing so, the use of performance to assess communicative ability could be better operationalized. Communicative competence is considered appropriate to encompass this, yet it does involve "opening Pandora's box". Since current testing models advocated by MEXT's Action Plan do not provide clear models to guide in the process of how to assess performance, it is of great concern if cultivating "Japanese with English abilities" is the goal. Thus, MEXT's Action plan ought to be revised, and align itself with models of communicative ability, supported by communicative competency theories. In reality, the complex nature of performance testing necessitates a huge undertaking, and it is questionable as to whether MEXT has the appropriate resources to ensure effective adoption of communicative testing. Yet if MEXT directives are to promulgate a rhetoric that seeks to cultivate English communicative ability among Japanese students, it is a step that should be taken.

References

Canale, M. & Swain, M. (1980). Theoretical bases of communicative approaches to second language teaching and testing. Applied Linguistics, 1, 1-47.

Celce-Murcia, M., Dornyei, Z., & Thurrel, S. (1995). Communicative competence: A pedagogically motivated model with content specifications. Issues in Applied Linguistics, 6 (2), 5 - 35.

McNamara, T. F. (1996). Measuring second language performance. London and New York: Addison Wesley Longman.

McNamara, T. F. (2000). Language testing. Oxford: Oxford University Press

MEXT. (2003, March 31). Regarding the establishment of an action plan to cultivate "Japanese with English abilities." Accessed April 13th, 2007 at http://mext.go.jp/english/topics/03072801.htm

O'Donnell, K. (2005). Japanese secondary English teachers: Negotiation of educational roles in the face of curricular reform. Language, culture and curriculum, 18 (3) 300 - 315.

Sage, K. & Tanaka, N. (2006). So what are we listening for? A comparison of the English listening constructs in the Japanese National Centre Test and TOEFL® iBT. In T. Newfields, I. Gledall, M. Kawate-Mierzejewska, Y. Ishida, M. Chapman, & P. Ross. (Eds.) Authentic Communication: Proceedings of the 5th Annual JALT Pan-SIG Conference. May 13-14, 2006. Shizuoka, Japan: Tokai University College of Marine Science. (p. 74 - 98). Retrieved Sept. 1, 2007 from http://jalt.org/pansig/ 2006/HTML/SageTanaka.htm

Suzuki, A. (2006, June 12). Nihon no Daigaku no Nyuushi-seido to Kyouiku Kakumei. [Japanese university entrance exams and educational reform]. Retrieved on August 22, 2007 from http://www.epis.com.hk/column/index.php?ID=23

[ p. 5 ]

Newsletter: Topic IndexAuthor IndexTitle IndexDate Index
Main Page Background Links Network Join
http://jalt.org/test/sag_1.htm
Main Page