Test construction

1. The most important task in pretesting is b. reviewing the items. Pretesting primarily focuses on ensuring that the test items are clear and effective before the actual testing occurs.
2. A distracter is considered to be b. ineffective if it has attracted more low than high examinees. Effective distracters should ideally attract a balanced mix of examinees based on their abilities.
3. The most important task in pretesting is b. planning the test. This involves careful consideration of how the test will be structured and what it aims to measure.
4. The degree of difficulty of the test can be controlled by changing the degree of b. homogeneity of the responses. A homogeneous set of responses indicates that all test-takers have a similar level of understanding, affecting overall difficulty.
5. b. Item discrimination refers to the power of distinguishing among the testees on the basis of their knowledge and language proficiency. This measure helps identify how well a test item differentiates between high and low performers.
6. One of the immediate benefits of item analysis is, as we have seen, to draw the attention of the test designer to items which contain d. problem or ambiguities in their construction which escaped notice during the construction of the test.
7. A distracter is mal-functioning if it attracts c. more low than high students. This indicates that it fails to serve its purpose effectively.
8. In a multiple choice test, we should not only determine the difficulty level and discrimination level, but also the effectiveness of the b. incorrect distracters. Evaluating distracters is crucial for ensuring they challenge all examinees appropriately.
9. The a. correlation coefficient can help the language teacher in determining the relationship between the various language skills, providing insights into how different skills interact.
10. The extent to which an item separates the high and low students is determined by the b. discrimination level. High discrimination levels indicate that the item effectively differentiates between varying levels of student ability.
11. If none of the students in an upper group answers the correct choice and all of the 20 students in the lower group answer it correctly, the item discrimination index will be a. -1.0 indicating a complete failure to discriminate between groups.
12. Item a. discrimination refers to the power to screen examinees on the basis of their proficiency, highlighting how well items can differentiate among varying levels of knowledge.
13. Thirty students were tested; 20 answered item 4 correctly, so the difficulty index for item 4 would be c. 0.67 (20 out of 30).
14. Item analysis is conducted in order to d. include items of appropriate facility and discrimination index in the test, ensuring that each item functions as intended within its context.
15. Of the following items, the one which is likely to be a hybrid item is the b. multiple-choice item, as it combines various question formats into one structure.
16. Achievement tests differ from progress tests in that they c. are syllabus-based and standardized, focusing on measuring knowledge against specific curriculum objectives.
17. A test item with a discrimination index of 0.82 and a facility index of 0.55 c. discriminates well and is at the right level of difficulty, indicating effective performance across different examinee levels.
18. The tests which are administered to determine one's progress towards instructional objectives of a training program are called b. achievement tests.
19. The attractiveness of alternatives in multiple-choice items is determined through d. choice distribution, which assesses how often each option is selected by examinees.
20. Item analysis is one of the stages of b. pretesting, providing valuable insights into item effectiveness before full-scale testing occurs.

Answers and Explanations

21. b. 20%

• Explanation: The facility value is calculated as the number of correct responses divided by the total number of responses. Here, 4050=0.85040​=0.8 or 80%, so the incorrect responses (10 out of 50) yield a facility value of 20%.

22. c. reflect the difference between L1 and L2

• Explanation: A plausible distracter should challenge test-takers by reflecting common errors or misconceptions between their first language (L1) and the target language (L2).

23. c. broken sentence

• Explanation: The sentence structure indicates that it is incomplete, making it a broken sentence item.

24. a. there is more than one correct answer

• Explanation: Both "alarm" and "watch" can be considered correct answers, making the item ambiguous.

25. c. 4

• Explanation: The formula for correction for guessing is Score=Correct−Incorrectn−1Score=Correct−n−1Incorrect​. Here, n=4n=4, so 15−453=15−15=015−345​=15−15=0.

26. b. contextual clues

• Explanation: The phrase "the missing girl was found with" provides contextual clues that make some options more obvious than others.

27. c. showing the correlation of an item with the others

• Explanation: Item analysis typically focuses on individual item performance rather than correlations with other items.

28. d. the students share the same native language

• Explanation: Contrastive analysis is effective when students have similar linguistic backgrounds, which aids in identifying specific language issues.

29. a. the testees in both groups

• Explanation: In the formula (Hc+Lc)/N(Hc+Lc)/N, N represents the total number of test-takers in both groups being analyzed.

30. c. had be

• Explanation: The correct construction should be "had been," making "had be" grammatically incorrect.

31. d. the item measures knowledge of both inflection and derivation

• Explanation: This is not a shortcoming; it indicates that the item assesses multiple aspects of language knowledge.

32. a. A=10, B=2, C=20, D=68

• Explanation: This distribution shows a clear majority selecting the correct answer (D), indicating good discrimination.

33. c. the correct choice depends on the testee's subjective judgment

• Explanation: The choice "speak" is objectively correct based on grammatical rules, not subjective judgment.

34. a. lexical difficulty of the stem

• Explanation: The vocabulary used in this item is appropriate for most test-takers and does not present lexical difficulty.

35. d. All distractors should not be of similar length and level of difficulty.

• Explanation: Ideally, distractors should be similar in length and difficulty to avoid giving away clues about the correct answer.

36. a. The item is multiple-choice; so the scoring is objective.

• Explanation: Multiple-choice items typically have clear right or wrong answers, making scoring objective.

37. c. pretesting

• Explanation: Item discrimination and facility values are typically assessed during pretesting to evaluate item effectiveness before finalizing a test.

38. a. as large as one

• Explanation: Maximum item facility (easy items) and maximum discrimination can theoretically reach a value of one but are generally not both achievable simultaneously.

39. c. are used to improve the quality of the test

• Explanation: Both concepts help refine tests by analyzing how well items perform and how effectively they differentiate between test-takers.

40. c. an item

• Explanation: The smallest independent unit of a test is referred to as an item, which can be a question or task within an assessment.

Answer

41. d. analysis
Item analysis refers to evaluating the effectiveness of test items, identifying their strengths and weaknesses, and understanding how they contribute to overall test performance 

42. b. 0.25
The item facility index is calculated as the proportion of testees answering correctly. Here, 624=0.25246​=0.25 

43. a. very easy
If more correct responses come from the lower group than the higher group, it indicates that the item is too easy, as lower-performing students are answering it correctly 

44. a. 0.75
The item discrimination index is calculated using the formula: Discrimination Index=Correct in High Group−Correct in Low GroupTotal in High GroupDiscrimination Index=Total in High GroupCorrect in High Group−Correct in Low Group​. Here, if all 45 in the high group answered correctly (45) and 15 in the low group answered correctly (15), then D=45−1545=0.75D=4545−15​=0.75 

45. b. The function of the test
Determining the function of the test is crucial as it guides the overall purpose and design of the assessment 46. c. nonfunctioning
A distracter that does not attract any students is termed a nonfunctioning distracter, indicating it fails to serve its purpose in multiple-choice questions 

47. b. meticulous pretesting
Necessary changes in directions and distracters are best informed by meticulous pretesting, which helps identify issues before finalizing the test 

48. b. draw up a detailed table of specifications
Creating a detailed table of specifications ensures that all content areas and skills are covered appropriately before writing test items 

49. b. do item analysis
The major aim of pre-testing items is to conduct item analysis, which assesses how well each item performs and contributes to overall test reliability 

50. d. be replaced by carefully constructed examples
Test instructions cannot be entirely replaced by examples; while examples can clarify, instructions are essential for guiding examinees on how to approach the test 

51. b. describe the nature of the language being tested
Understanding the nature of the language being tested is fundamental before constructing a test to ensure relevance and appropriateness 

52. b. The item is at the right level of difficulty and discriminates well.
An item with a discrimination index of 0.75 and a facility value of 0.52 indicates it effectively distinguishes between high and low performers while being appropriately challenging 

53. c. The item is too difficult and discriminates in the wrong direction.
With an index difficulty of 0.25 and a negative discrimination index (-0.10), this suggests that lower-performing students are doing better on this item than higher-performing ones, indicating poor quality 

54. c. Planning, writing, reviewing, pretesting, and validating
This sequence outlines a systematic approach for constructing a test effectively, ensuring thorough evaluation at each stage 

55. a. planning the test content and design
Planning is critical as it lays the foundation for all subsequent steps in test construction, making it both important and time-consuming 

56. d. determining the general course objectives
Establishing course objectives is essential as it aligns the test with educational goals and expected outcomes for learners 

57. c. write more items than needed
Writing more items than necessary allows for selection during finalization, ensuring that only high-quality items are included in the test 

58. d. pretesting
Pretesting helps identify defective or inappropriate items by evaluating their performance with actual testees before final administration 

59. a. allow all examinees to begin the problems on an equal footing
Test directions serve to standardize understanding among all examinees, ensuring fairness in interpretation and execution of tasks 

.60. b. submit the items to a colleague
Submitting items for peer review during revision helps catch errors or biases that may have been overlooked by the original writer 

.

Answer

61. d. validity of the test
    Pretesting primarily checks aspects like test directions, required time, and item difficulty, but it does not assess the validity of the test itself.
62. a. increasing level of difficulty
    When assembling multiple-choice items, they should be ordered by increasing difficulty to facilitate a smoother testing experience for students.
63. b. the answer positions do not form any observable pattern
    In creating the final form of multiple-choice items, it's crucial to ensure that answer positions are randomized to avoid giving clues about correct answers.
64. b. discriminability
    The degree of difficulty can be estimated by examining the discriminability of responses, which indicates how well items differentiate between high and low performers.
65. d. 0.3 and 0.7
    Test constructors often accept items with facility values between 0.3 and 0.7, indicating a range where questions are neither too easy nor too difficult.
66. a. remove the weak points
    Test analysis allows for the identification and removal of weak points in test items even before recording results.
67. d. multiple-choice
    Item analysis is most commonly associated with multiple-choice tests, as it helps evaluate the effectiveness of each question.
68. a. whether or not the questions discriminate
    Through item analysis, one can determine whether questions effectively discriminate between different levels of student ability.
69. b. students who got each question right
    The level of difficulty for each question is defined by the percentage of students who answered it correctly.
70. c. poor discrimination
    Weak distracters often lead to poor discrimination in test questions, making it hard to differentiate between student abilities.
71. b. the index of difficulty
    The proportion of students answering an item correctly is indicated by the index of difficulty, which quantifies how challenging an item is.
72. b. using tests which can be scored objectively
    Examiner variability can be minimized by employing tests that are scored objectively, reducing subjectivity in grading.
73. c. the table of specification
    The main purpose of a table of specifications is to ensure that a test includes a representative sample of course materials.
74. a. Planning succeeds item writing
    In test construction, planning occurs before item writing, establishing a framework for what to include in the test.
75. c. The form of the test
    The form of the test is not typically considered when determining its function; instead, factors like examinee characteristics and specific purposes are prioritized.
76. b. stem
    The stem of an item serves to elicit information from examinees by presenting the question or problem they need to address.
77. a. function
    The function of an item refers to the specific information that is intended to be elicited from testees through their responses.
78. a. distractors
    Distractors are different from choices, options, and alternatives as they specifically refer to incorrect answer options in a multiple-choice format.
79. b. its stem
    The lead of an item typically refers to its stem, which presents the main question or prompt for respondents.
80. d. the way a test is scored
    Objectivity and subjectivity pertain to how a test is scored; objective tests have clear right or wrong answers, while subjective tests involve personal judgment in scoring.

Answer

81. Completion items: These are items that require examinees to supply the missing part(s) of the stem or finish an incomplete stem. This type of question is designed to assess recall and understanding of the material

82. Form: In psycholinguistic classification, the linguistic dimension primarily deals with the form of language, which encompasses the structure and organization of linguistic elements

83. Planning: The first step in test construction is planning, where objectives and content are determined before any writing or reviewing takes place

84. Correct order of steps: The systematic construction of a test follows this order: planning, writing, pretesting, reviewing, and validating. This sequence ensures that each phase builds on the previous one for effective test development

85. Facility index: For an item to remain in a test, it should have a facility index between .33 and .67. This range indicates that the item is neither too easy nor too difficult for most test-takers

86. Item characteristics: Item characteristics include item facility, item discrimination, and relevance. These factors help evaluate how well an item performs in assessing knowledge or skills

87. True statement about item facility and discrimination: The correct statement is that an item with too high or too low facility is likely not to have good discrimination power. This indicates that both metrics are interrelated in evaluating item effectiveness

88. Level of discrimination: The level of discrimination of a test item indicates the difference between high and low students, reflecting how well an item can differentiate between varying levels of student performance

89. Not included in item analysis: The characteristic that is NOT included in an item analysis is reliability. Item analysis typically focuses on difficulty, discrimination, and distractor evaluation

90. Planning refers to: In test construction, planning refers to determining the function of the test, which involves setting clear objectives for what the test aims to assess

91. Ideal unity in NRT system: The ideal characteristic for unity in a Norm-Referenced Test (NRT) system is related to item discrimination, which ensures that items effectively differentiate between varying levels of student ability