Shiken: JALT Testing & Evaluation SIG Newsletter
Vol. 12 No. 1. Jan. 2008. (p. 19, 20) [ISSN 1881-5537]
PDF PDF Version

Assessment Literacy Self-Study Quiz #4
by Tim Newfields

This column mentions a range of questions about testing, statistics, and assessment in quiz format to help readers become more familiar with some concepts in this evolving field and consider current testing practices. Suggested answers for the items below are online at http://jalt.org/test/SSA4.htm.

Part I: Open Questions

  1. One Japanese university English entrance examination included this task:

    INSTRUCTIONS (translated from Japanese): Read the sentence below, then select
    one sentence (A-D) in which the term "that" is used in the same manner as in the original sentence:

    Sentence: 
    If there is something that you want to see, let me know.

    Possible Answers: 
         	(A) The shoes that I bought did not fit.
     	(B) She hid the fact that she spoke French.
     	(C) I'm so glad that he could come.
     	(D) Such was her anger that her face turned red.

    Question: What is this test item likely measuring? What would be the arguments for and against including this item on an EFL test for aspiring college students?

  2. On October 9, 2006 a short article in The Washington Post mentioned a statistic supporting current testing policies in American schools. Based on an unspecified report by Public Agenda, they claimed that "71% [of the students surveyed] felt the number of tests they have to take is 'about right'" and that "79% thought standardized test questions are fair."

    Mention at least three problems with this newspaper report and at least four things that should have been done to present the information ethically.

    Source: The Washington Post. (2006). Too much testing? Retrieved February 10, 2007from www.washingtonpost.com/wp-dyn/content/article/2006/10/09/AR2006 [Inactive URL].

  3. Mention one example of each of the following types of hypotheses in the field of foreign language study: (1) a positive directional hypothesis, (2) a negative directional hypothesis, and (3) a non-directional hypothesis.

  4. To estimate the English vocabulary size of a group Japanese examinees, a bilingual test of vocabulary was devised. In this format, six English words appear on one line. Above that, two of those words are defined in Japanese, as in this example:
    Sample J/E Vocab. Test Item


    Source: Mochizuki, M. (2998) . Nihonjin gakusha no tameno goi saizu tesu [A vocabulary size test for Japanese learners of English]. Gogaku Kennkyuujo Kiyou, 12, 27-53. Cited in K. Kasahara. (2006) Producing a revised version of the Mochizuki Vocabulary Size Test. JLTA Journal, 9, 55 - 72.

    Any problems with this type of test design as a measure of English vocabulary?

  5. Recently the draft version of an article in the JALT 2007 Pan-SIG Proceedings explored how classroom performance on a composite 98-item in-house EFL test correlated with a 5-point Likert scale assessment of how students reported liking their class. 59 Japanese high school EFL students participated in their survey. Those who scored 35 points or less (n=14) on the composite tests were placed in a "low" sub-group. Those with 36-65 points (n=37) on the composite tests were placed into a "mid" sub-group. Those with a combined score of 66 points or higher (n=8) were placed in a "high" sub-group. The results are summarized in Figure 1, with small circles indicating the mean scores for each sub-group and anchor bars claiming to indicate "one standard deviation" above and below each mean.
    Hamada, Oda, and Kito's Figure 1
    First, this graph (which has seen been revised) has one mistake that calls into question the validity of all the material. Can you recognise it? Then look at this graph and suggest any ways to make the information clearer. Next, without reading the article, briefly interpret the graph. What can be surmised about the distributions of the sub-groups? Finally, suggest a viable alternative way of comparing the ratings of the high, mid, and low sub-groups.

[ p. 19 ]

Source: Hamada, Y., Oda, S., Kito. K. (2007). Correlations between Japanese high school students' language learning beliefs, self-reported attitudes towards group dynamics, and performance in two in-house EFL tests. In T. Newfields, I. Gledall, P. Wanner, & M. Kawate-Mierzejewska. (Eds.) Second Language Acquisition - Theory and Pedagogy: Proceedings of the 6th Annual JALT Pan-SIG Conference. May. 12 - 13, 2007. Sendai, Japan: Tohoku Bunka Gakuen University. (pp. 42 - 55) Retrieved January 15, 2008 http://jalt.org/pansig/2007/HTML/HOK.htm

[ SUGGESTED ANSWERS ]

Part II: Multiple Choice Questions

  1. A t-distribution is generally resembles a normal Z-distribution except


  2. Which of the following is generally not considered a test method facet?


  3. Quasi-experimental research designs (i.e. studies in which the subjects are not randomized into different experimental and control groups) are also often known as

  4. In statistics, the symbol tc sometimes refers to


[ SUGGESTED ANSWERS ]
NEWSLETTER: Topic IndexAuthor IndexTitle IndexDate Index
TEVAL SIG: Main Page Background Links Network Join
last Main Page next
HTML: http://jalt.org/test/SSQ4.htm   /   PDF: http://jalt.org/test/PDF/SSQ4.pdf
ASSESSMENT LITERACY ONLINE SELF-STUDY SERIES:

Quiz #1: Qs   / As   *    2: Qs   / As   *    3: Qs   / As     *    4: Qs   / As   *    5: Qs   / As   *    6: Qs   / As   *    7: Qs   / As   *    8: Qs   / As   *    9: Qs   / As   *    10: Qs   / As   *    11: A HREF="SSQ11.htm">Qs   / As  

[ p. 20 ]