PDF Version
Some challenges in academic researchStephen A. Templin (Meio University) |
I would like to mention a few things about the quality of research underscored in my parody on the Templin 1/2k. Although that parody was written for fun to provide a good chuckle, it also touches on some important issues relating to academic research. Two of these issues I find somewhat disturbing, and two are significantly more disturbing.
Somewhat Disturbing
| "much research in Japan seems to exhibits little, or no, literature review." |
The first issue, about gaining funds for research, has not caused me insurmountable difficulties. I have always seemed to scrape together enough money from one source or another to do the research I wanted, and in general more research funds are available in Japan than other countries in Asia. But even though I have been able to allocate research funds for research, it is regrettable that much research in Japan seems to exhibits little, or no, literature review. As a result, a lot of money being wasted trying to reinvent the wheel because proper research reviews were not conducted. Also, too often research projects in Japan do not give adequate thought as to how answer the research questions they raise. It is surprising how many research questions are worded in vague ways which make any meaningful answers difficult.
Another issue – the exploitation of students and/or junior faculty members in research – has not impacted my work in a huge, negative way. Exploitation of students and junior faculty for research occurs in the U.S. and other parts of the world: sometimes the people who do the most work get the least credit. I feel this happens too often in Japan, where authors are frequently listed in order of rank rather than by order of how much each person contributed to the study.
More Disturbing
| "the people on most university entrance exam committees have little or no knowledge of testing principles such as reliability or validity" |
The two more serious issues are the frequent lack of reliable and valid research in Japan, and the lack of predictive validity in our field as a whole. In this country, checking the reliability and validity of a test (Brown and Yamashita, 1995; Murphey, 2000) or questionnaire (Griffee, 1997) is the exception rather than the norm. This is particularly frustrating in the situation of university entrance exams, where the stakes are high. Even though students' futures are often determined by which university they gain admission to, the people on most university entrance exam committees have little or no knowledge of testing principles such as reliability or validity. Without the knowledge of such principles, these committees do not know how to make consistent, accurate decisions in accepting and rejecting students. As a consequence, most Japanese universities claiming to test English proficiency are really only testing translation skills (yakudoku) (Gorsuch, 1998), knowledge of syntactic rules, test-taking skills, or other factors.
Recently, faculty development (FD) has become a buzzword in Japan, and people talk about improving teaching, but without reliable and valid tests, it is difficult to judge the effectiveness of classroom teaching or a curriculum as a whole. Universities form committees to create faculty evaluations, but I think most committees do not know how to make their questionnaires reliable and valid. The results of many classroom questionnaires are often a matter of chance and do not actually evaluate good teaching.
[ p. 10 ]
Consequently, it is difficult to judge the effectiveness of teachers and make fair decisions in promotions, etc. I believe that even when committee members do know how to make reliable and valid tests and questionnaires, they cannot get the necessary support from administration because few administrative personnel are trained in this area. Although many researchers who know how to create reliable and valid tests and questionnaires exist in Japan, they are seldom well represented in committees that create university entrance exams or faculty evaluations — and even when they are represented, they may not be able to acquire the necessary support from their universities' administrative staff.
Finally, the issue that probably frustrates me most is the lack of predictive validity in second language (L2) research – not just in L2 research, but in other fields such as psychology, as well. Our field is full of studies that describe and explain what learners do (Larsen-Freeman, 1991), but lacking in studies that can predict language learning. Few people would want to fly in a plane that people could describe and explain but no one could predict whether it could stay in the air or not – and yet we, as language teachers, fly in this plane every time we step into the classroom. Although we are inundated with many "theories" in L2 research, they are not theories in the scientific sense. In order for something to truly be a theory, it must be able to describe, explain, and predict. And teachers should be able to use the theory to help learners learn languages (Goodson, 1993). Although we are faced with a number of challenges in academic research, I feel that predictive validity is our biggest challenge. In spite of my parody and the challenges stated above, I am optimistic that we will become more successful in helping students learn languages through academic research in Japan and the field of language teaching as a whole.
References
Author IndexTitle IndexDate Index
Background
Links
Network
Join
[ p. 11 ]