34. Which statement is NOT TRUE regarding the notion of "criterion" in CRT? 1) The term criterion refers to the standard, called a criterion level, against which each student's performance is judged.

2) If the cut-point for passing a CRT is set at 70%, that is the criterion level. 3) The notion of criterion in CRT has been well-defined in relevant literature 4) The material that the student is supposed to learn in a course is the criterion against which he or she is being measured.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

Characteristic	Norm-Referenced Tests	Criterion-Referenced Tests
Type of Interpretation	Relative (A student’s performance is compared to that of all other students in percentile terms.)	Absolute (A student’s performance is compared only to the amount, or percentage, of material learned.)
Type of Measurement	To measure general language abilities or proficiencies	To measure specific objectives-based language points
Purpose of Testing	To spread students out along a continuum of general abilities or proficiencies	To assess the amount of material known, or learned, by each student
Distribution of Scores	Normal distribution of scores around a mean	Varies, usually non-normal (students who know all of the material should all score 100%)
Test Structure	A few relatively long subtests with a variety of question contents	A series of short, well-defined subtests with similar question contents
Knowledge of Questions	Students have little or no idea what content to expect in questions	Students know exactly what content to expect in test questions

In terms of the type of interpretation, one essential difference between these two categories of tests is that each student’s performance on a CRT is compared to a particular criterion in absolute terms. Some confusion has developed over the years about what the criterion in criterion-referenced testing refers to. This confusion is understandable because two definitions have evolved for criterion. For some authors, the material that the students are supposed to learn in a particular course is the criterion against which they are being measured. For other authors, the term criterion refers to the standard, also called a criterion level or cut point (see Chapter 10), against which each student’s performance is judged. For instance, if the cut-point for passing a CRT is set at 70 percent, that is the criterion level.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (2005, p. 3)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - پاسخ: کریم وفائی سرشت kvafaee @:Channel Telegram - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

35. How can dictation be described in terms of its channel and mode? 1) It is written channel and oral channel, as well as receptive mode and productive mode. 2) It is partially written channel and mostly oral channel, as well as partially receptive mode and mostly productive mode.

3) It is partially written channel and partially oral channel, as well as partially receptive mode and partially productive mode.

4) It is mostly written channel and partially oral channel, as well as mostly receptive mode and partially productive mode.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

Consider also the commonly used dictation task. Dictation is sometimes used in language programs to test listening ability. Given the nature of the distinctions discussed above, what channel(s) are involved in taking a dictation? And what mode(s)? The student must understand what is being dictated, thus utilizing the oral channel, but must also be able to write it down, which is tapping the written channel. The student must receive the message, which is receptive mode, and must put it on paper, which is productive mode. After considering the dictation as it relates to these different issues, is it still logical to view it as a test of listening ability, or is it perhaps something more complex?

Thus, thinking about what is actually happening in language tests can be enhanced by thinking in terms of the productive and receptive modes and the written and oral channels. Remember, however, that various combinations are possible. For instance, a test like dictation is best described as partially written channel and partially oral channel, as well as partially receptive mode and partially productive mode. Hopefully, knowing about these issues will help teachers to better understand what they are testing.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (1996, p. 28)

36. Which item can best describe dictation and cloze test in terms of their objectivity and subjectivity?

2) Based on the context, the tests can be classified as subjective or objective. 3) They cannot be classed as entirely objective nor completely subjective. 4) They can be classed as mostly objective and partially subjective.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

We also have cloze and dictation subtests, which cannot be classed as entirely objective (because some judgments must be made) nor completely subjective (because the range of possibilities for those judgments is fairly restricted).

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (1996, p. 32); James Dean Brown (2005, p. 27)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

The calculation of IF for any item will follow a consistent pattern. Consider Item 3. Count up the number of students who answered Item 3 correctly (7); then, count the number of people who took the test (16), fill in the formula, and do the calculations:

With this simple IF index in hand, the teacher knows that about 44 percent of the students answered Item 3 correctly. Try calculating the IF for a few of the other items shown in Table 4.1. The answers are shown in Screen 4.5 on page 74.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (2005, p. 68)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

Another statistic that is often used for the same purpose as the ID is the point biserial correlation coefficient. This statistic is usually lower in magnitude when compared directly with the ID for a given item but is analogous in interpretation. Because ID is easier to calculate and understand conceptually, teachers are much more likely to use it in most language programs. Hence, I can

safely delay the discussion of the point biserial correlation coefficient until Chapter 6. - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (2005, p. 70)

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

A ratio scale also represents the ordering of a named group of data and shows the distances between the points in the rankings, but it provides additional information. First, a ratio scale has a zero value; and second, as the name implies, the points on the scale are precise multiples, or ratios, of other points on the scale. For instance, if the lights in a room are turned

off, there is zero electricity flowing through the wires. If I turn on a 50- watt bulb, a certain amount of electricity is flowing through the wires. If I then switch on another bulb that uses 100 watts, two times as much electricity is flowing through the wires. Thus, electricity can be measured on a ratio scale; zero electricity makes sense as do multiples or ratios along the scale.

However, arguing that any person knows zero, or no part, of any foreign language would be a difficult position to take. Even a person who has never studied a foreign language knows certain lexical, phonological, and syntactic facts about language in general from learning a native language. This information, in addition to providing cognates and other links between any two languages, can be brought to bear on the task of learning a foreign language. Hence, the position that a person knows zero Japanese (or any other foreign language) is theoretically untenable.

Another shaky position would be to state that a student who scores 100 on a Russian proficiency test knows twice as much Russian as another student who scored 50, or that the

student who scored 50 knows five times as much as a student who scored 10. Ratio scales of concern to language teachers include things like the students’ ages, the number of years of schooling that they have had, their years of language study, the number of languages they speak, and so forth.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (1996, p. 97)

40. The standard deviation is usually better than the range because------------. 1) The SD is more applicable than the range.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

The standard deviation is an averaging process, and as such, it is not affected as much by outliers as the range. Consequently, the standard deviation is generally considered a stronger estimate of the dispersion of scores.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (2005, p. 102)

1) To calculate a student's z score, first, subtract the mean from the student's score; then divide the result by the variance for the test

2) The Z score will always be plus (+) if the student scored above the mean and minus (-) if the score was below the mean.

3) A z score indicates the number of standard deviations that a student's score falls away from the mean.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - Explanation

The z score is a direct indication of the distance that a given raw score is from the mean in standard deviation units. The z score for each student can be calculated on any test by using the following formula:

In other words, to calculate a student’s z score, first subtract the mean from the student’s score; then divide the result by the standard deviation for the test. If a student scored 51 in the distribution shown in Figure 6.5 (p. 121), where M = 41 and S = 10, the z score for that student would be as follows:

This student’s z score would be +1.0, or one standard deviation unit above the mean. If another student scored 21 raw score points on the same test, that student’s z score would be:

In short, a z score indicates the number of standard deviations that a student’s score falls away from the mean. This value will always be plus (+) if the student scored above the mean, minus (- if the score was below the mean, and zero if the student scored right on the mean. Note that z scores seldom turn out to be perfectly round numbers like those found in the examples above. These were used so that the demonstration would be clear. In fact, uneven z scores, like +1.67, 0.71, or -3.13, are much more likely to occur in real test data. Nevertheless, the steps involved in calculating z will be exactly the same.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (2005, pp. 123-125)

42. --------- is one way of looking at the degree to which the curve in the middle of a distribution is steep or the degree to which the distribution is peaked. 1) Ceiling effect

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - -

Kurtosis is one way of looking at the degree to which the curve in the middle of a distribution is steep, or the degree to which the distribution is peaked. If the height of the peak, relative to the width, is too different from what would be expected in a normal distribution, that is, either too

peaked (referred to as leptokurtic) or too flat (referred to as platykurtic), problems may arise in applying testing statistics. Hence testers should always check for this condition. Simple inspection of a histogram will reveal the degree to which the distribution appears to have a normal shape, or depart from that shape.

- - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - See: James Dean Brown (2005, p. 132)

43. Which item does NOT refer to the possible problems with correlational analysis? 1) Sample inadequacy