Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 11 No. 2. Sep. 2007. (p. 19, 20) [ISSN 1881-5537]
PDF PDF Version

Assessment Literacy Self-Study Quiz #3
by Tim Newfields

This column mentions a few basic questions about testing, statistics, and assessment in a quiz format to help readers become more familiar with some of the jargon and concepts in the field and also think critically about some testing practices. Suggested answers for the items below are available at http://jalt.org/test/SSA3.htm.

Part I: Open Questions

  1. What statistics should be made public about a commercial test? If too little information is offered, a loss of face validity might occur. On the other hand, general readers might be baffled by too much information. Take a look at what was offered to the public about some recent TOEIC® scores from the Institute for International Business Communication website:

    The number of people who took this test in July 2007: 57,131
    Listening Sec. Reading Sec. Total
    Maximum Score 495 495 990
    Minimum Score 5 5 10
    Mean Score 316.0 264.1 580.1
    Standard Deviation 86.9 92.4 170.4


    Source: IIBC. (2007). Koushiki Deeta: Heikin Skoa Shosai. [Public Information: Mean Score Details].
    Retrieved on August 10, 2007 from http://www.toeic.or.jp/toeic/data/data_ave01.html?score_id=0000000018


    What other statistics about this test should probably be mentioned to the public?

  2. A teacher notices an error in her school entrance exam after the exam was administered: one multiple choice question had two possible correct answers. What, if anything, should the teacher do?

  3. Recently a rating was developed by Isao Kobayashi of Kawai Juku to determine the "Good Places to learn English" [Eigo no Yoi Mon] in Japan based on his personal analysis of an unspecified number of 2005 university entrance exam problems. Kobayashi concluded that the Osaka University of Foreign Studies and Niigata University were the best places to learn English based on the quality of their entrance exam questions. By contrast, Sophia University was ranked at 50th place because that author considered their English entrance examination questions poor.

    Any problem with this ranking system?

    Source: Kobayashi, Isao (2006). Jukensei ga Ochi-iriyasui Machigatta Eigo Gakushuu-hou wo Tadasu. In K. Shimizu (Ed.) 2007 Nen Daigaku Rankingu. Tokyo: Asahi Shinbun. p. 60-63.

  4. At a university in Japan a placement test was developed to stream science majors into different classes for an EFL reading program. Examinees read a 369 word passage about Alfred Nobel, then answered a series of true/false statements about the passage. One statement from that test was –

    [ p. 19 ]


    "Alfred Nobel lived his whole life in Sweden."

    The response format was to circle either "true" or "false" in the answer sheet. Any problematic points concerning this question? Also, any issues with using T/F response formats for this sort of test?

    Source: Weaver, C. (2007). A Rasch-based evaluation of the presence of item bias in a placement examination designed for an EFL reading program. In T. Newfields, I. Gledall, P. Wanner, & M. Kawate-Mierzejewska (Eds.). Second Language Acquisition - Theory and Pedagogy: Proceedings of the 6th Annual JALT Pan-SIG Conference. Retrieved September 21, 2007 from http://jalt.org/pansig/2007/HTML/Weaver.htm

  5. To compare the mean of a particular sub-group to the mean of a larger group that is within the same population, what test(s) should be performed?
[ SUGGESTED ANSWERS ]

Part II: Multiple Choice Questions

NOTE: At least one of these questions has more than one possibly "correct" answer, so be careful!
  1. Which of the following is not a common procedure to investigate test reliability?


  2. When reporting test scores to students, which of the following is/are unethical?


  3. To determine a EFL student's progress toward mastery of a classroom content area, a should be used.

  4. What does this statistical symbol denote: R2?


[ SUGGESTED ANSWERS ]
NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join
last Main Page next
HTML: http://jalt.org/test/SSQ3.htm   /   PDF: http://jalt.org/test/PDF/SSQ3.pdf
ASSESSMENT LITERACY ONLINE SELF-STUDY SERIES:

Quiz #1: Qs   / As   *    2: Qs   / As   *    3: Qs   / As   *    4: Qs   / As   *    5: Qs   / As   *    6: Qs   / As   *    7: Qs   / As   *    8: Qs   / As   *    9: Qs   / As   *    10: Qs   / As   *    11: A HREF="SSQ11.htm">Qs   / As  

[ p. 20 ]