Test Construction
an essay by
Delano P Wegener, Ph.D.
Rational for Tests:
Measures of student performance (testing) may have as many as five purposes:

  • Student Placement,
  • Diagnosis of Difficulties,
  • Checking Student Progress,
  • Reports to Student and Superiors,
  • Evaluation of Instruction.

Unfortunately the most common perception is that tests are designed to statistically rank all students according to a sampling of their knowledge of a subject and to report that ranking to superiors or anyone else interested in using that information to adversely influence the student's feeling of self-worth. It is even more unfortunate that the perception matches reality in the majority of testing situations. Consequently tests are highly stressful anxiety producing events for most persons.

All too often tests are constructed to determine how much a student knows rather than determining what he/she must learn. Frequently tests are designed to "trap" the student and in still other situations tests are designed to insure a "bell curve" distribution of results. Most of the other numerous testing designs and strategies fail to help the student in his learning process and in many cases are quite detrimental to that process.

In a Mastery Based system of instruction the two main reasons for testing are to determine mastery and to diagnose difficulties. When tests are constructed for these purposes, the other four purposes will also be satisfied. For example, consider a test which requires the student to demonstrate mastery and at the same time rigorously diagnoses learning difficulties. If no difficulties are indicated, it may be safely assumed that the learner has mastered the concept. That information may then be used to record student progress and to make reports to the student and superiors. Examining student performance collectively for a group of students provides information about the quality of instruction. Examining a single student's performance collectively for a group of learning objectives may be used to determine proper placement within that group of learning objectives.

It is therefore important that the instructional developer construct each question so that a correct response indicates mastery of the learning objective and any incorrect response provides information about the nature of the student's lack of mastery. Furthermore, each student should have ample opportunity to "inform" the instructor of any form of lack of mastery. Unfortunately the mere presence of a test question influences the student's response to the question. The developer should minimize that influence by constructing questions which permit the student to make any error he would make in the absence of such influence. For example, a multiple choice question should have all the wrong answers the student might want to select and should also have as many correct answers as the student might want to provide.

True/False Questions:
True/false questions should be written without ambiguity. That is, the statement of the question should be clear and the decision whether the statement is true or false should not depend on an obscure interpretation of the statement. A true/false question may easily be used, and most commonly is used, to determine if the student recalls facts. However, a true/false question may also be used to determine if the learner has mastered the learning objective well enough to correctly analyze a statement.

It is important to be aware that only two choices are available to the student and therefore the nature of the question gives the student a 50% chance of being correct. A single True/False question therefore is helpful only if the student answers the question incorrectly and the incorrect response indicates a specific misunderstanding of the learning objective. A collection of true/false questions, about a single learning objective, all answered correctly by a student is a much stronger indication of mastery. It is therefore important that the instructional developer construct a "test bank" containing a large number of true/false questions. It is also important to include numerous true/false questions on any test which utilizes true/false questions.

Ideally a true/false question should be constructed so that an incorrect response indicates something about the student's misunderstanding of the learning objective. This may be a difficult task, especially when constructing a true statement. The instructional developer should try to accomplish the ideal, but should recognize that in some instances he/she will not reach that goal.

Multiple Choice Questions:
Multiple choice questions should be written without ambiguity. That is, the statement of the question stem should be clear and should leave no doubt about how to select choices. Additionally the choices should be written without ambiguity and should contain all information required to make a decision whether or not to choose it. The decision whether to select or not select a choice should not depend on an obscure interpretation of either the stem or the choice. A multiple choice question may easily be used to determine if the student recalls facts. However, a multiple choice question may also be used to determine if the student has mastered the learning objective well enough to correctly analyze a statement.

The instructional developer should not construct multiple choice questions with a uniform number of choices, a uniform number of valid choices, or any other recognizable pattern for construction of choices. Instead the instructional developer should include as many valid and invalid choices as is required to determine the student's deficiencies with respect to the learning objective. Moreover, each choice should appear to be a valid choice to some student.

Multiple choice questions should therefore contain any number of choices with one or more valid choices. The student is of course required to select all valid choices and failure to select any one of the valid choices will provide information about the student's misunderstanding of the learning objective in the same way that selection of an invalid choice reveals the nature of his/her misunderstanding.

The nature of the choices provided in a multiple choice question may be of two types: those which require merely recall of facts and those which require additionally activity such as synthesis, analysis, computation, comparison, or diagramming. The instructional developer who is seriously concerned with the student's success will use both types extensively.

Fill-in-the-Blank Questions:
The temptation, when constructing fillintheblank questions, is to construct traps for the student. The instructional developer should avoid this problem. Ensure that there is only one acceptable word for the student to provide and that the word (or words) is significant. Avoid asking the student to supply "minor" words. Avoid fillintheblank question with so many blanks that the student is unable to determine what is to be completed.

Sometime/Always/Never Questions:
The collection of Sometime/Always/Never (referred to as SAN) statements are statements which are: true sometimes, always true, and never true. The statements used in these questions must be stated carefully and should contain enough information to permit the student to decide whether the statement is true sometimes, always, or never.

SAN questions (especially the sometimes statements) are the most difficult to construct but can be the most significant part of a test. SAN questions should be constructed to force the student to engage in some critical thinking about the learning objective. When used properly, SAN questions force the student to consider important details about the learning objective. Careful use of this type of question and careful analysis of student's response will provide detailed information about some of the student's deficiencies.

SAN questions are especially appropriate, and easy to construct, for learning objectives addressing concepts which are "black" or "white" except in a few cases. The true statements in a collection of true/false questions are of course always true statements while the set of false statement may be further subdivided into those which are true sometimes and those which are never true.