On a Microsoft Access test, I had the experience of having all 18 of my students miss the same question.
Original
Objective:
After completing this project, the student should be able to filter records.
Evaluation Tool:
To filter records based on values in more than one field, click the ______________ button on the toolbar.
(The correct answer to the question is Filter by Form; the answer given by every student was Filter by Selection which is what they actually practice in the project.)
I have determined that the objective and evaluation tool are reliable (yielding consistent results), but not valid (does not measure what the students should be able to do). I propose the following two methods to correct the problem:
Correction #1
Objective:
By the end of this lesson, given a computer and appropriate software, a demonstration and practice, the student will filter records based on values in more than one field.
Evaluation Tool:
Apply a filter to the datasheet so that only customers living in CA and having a credit limit of $500 appear. Print one copy of the filtered datasheet.
Correction #2
Objective
By the end of this lesson, given a list of methods for filtering records, their definitions, a lecture and practice, the student will write a short description for each method using his or her own words.
Evaluation Tool:
In your own words, write a short description of Filter by Form and Filter by Selection.
Discussion:
The original objective was in the psychomotor domain and the assessment tool in the cognitive domain. In addition, the objective was not specific enough to be accurately measured.
In Correction #1, I have rewritten the original objective to be more specific but still within the psychomotor domain. The evaluation tool has rewritten as a Level 3 guided response assessment.
In Correction #2, I have again rewritten the original objective to be more specific but within the cognitive domain. The evaluation tool has been rewritten as a Level 2 comprehension assessment.
I believe both corrections result in valid assessment tools because they measure what the objective states the student should be able to do. It is uncertain whether the revised assessment tools will be reliable; only time will tell!