Variations
of Conventional Multiple Choice Question Formats
1)
conventional matching (two-column)
2)
unconventional matching (three-column): adds a third
column. The
second and third columns’ answer choices are
staggered (i.e., “1” in first column, “2” in
second, “3” in first, “4” in second---or
“a” in first, “b” in second, etc.)
3)
pick the closest—used for number questions, a
standard set of numbers is presented and students pick
the closest to their answer.
Avoids the tendency in math questions to use
the supplied options to pick the correct answer.
d)
uncued---uses a large set of distractors (sometimes in
the hundreds) to virtually eliminate guessing.
Alternate-Choice
Items
These
have only two options, but they are NOT true-false
items (because unlike T/F, they offer a comparison
between two choices whereas T/F says whether one
choice is right or wrong).
A good choice for items for which it is
difficult to write more than one plausible distractor.
Provides for higher reliability because you can
ask more items in a given time period.
Especially good for high-achieving students who
are skilled at eliminating less plausible distractors.
The
biggest problem with alternate choice is that there
are only two choices, which means even a guesser is
likely to score about 50% (vs. 25% for four-choice MC
tests). Thus
the range of the test scores is truncated from 50% to
100%.
True-False
Items
Advantages:
Multiple
Mark (Multiple-Multiple-Choice)
In
this variation of the Multiple True False format,
students mark the choice if it is true and do NOT mark
it if it is false.
Whereas with standard MTF format there is a
bias toward guessing “True”, here there is a bias
toward omission (i.e., guessing “False”.
Context-Dependent
Item Sets (aka bundles, scenarios, problem sets,
testlets)
Here,
one presents introductory information followed by a
series of questions that depend on that information.
This
format offers the potential of measuring higher-order
processes and is increasingly popular, but has not
been well-studied.
It has several variants:
a)
comprehension: presents a literary excerpt and asks
several questions about it.
b)
problem-solving: here, each item builds on the
preceding one or asks about the next step in the
problem-solving process.
c)
interlinear: good for measuring writing skills, this
variant embeds several items within a paragraph,
asking students to choose the best alternative for
each word or phrase.
d)
graphical: e.g.,
presenting a table or graph and asking several
questions about its interpretation.
Dangerous
Answers
While
rarely the primary focus of a question, this argues
for incorporating distractors identifying dangerous or
incompetent behaviors.
This shows promise on certification tests where
practitioners deal with the public and errors could
cause harm (e.g., medical professions).
Item
Shells
Shells
are “hollow” items containing the syntax for the
general class of items.
“Which
is an example of (any concept)?”
a.
Example
b.
Plausible non-example
c.
Plausible non-example
Steps
in Developing Item Shells
1.
Start with the stem of a successful item.
“What
is the distinguishing characteristic of hydrogen?”
2.
underline key words or phrases representing the
content of the item
“What
is the distinguishing characteristic of hydrogen?”
3.
Identify alternate words or phrases for each key word
or phrase.
(any
other gases studied in this unit---e.g., oxygen,
argon, carbon dioxide, carbon monoxide)
4.
Select a variables from the range.
“Oxygen”
5.
Write the item stem.
“What
is the distinguishing characteristic of oxygen?”
6.
Write the correct answer.
“It
is the secondary element in water”
7.
Write up to four plausible distractors.
“It
has a lower density than hydrogen”
“It
can be fractionally distilled”
“It
has a lower boiling point than hydrogen”
*************************************************
Item
Modeling
Item
modeling is derived from scenarios constructed by
experts and is used to test the same types of
higher-level thinking that would actually be used by
professionals in the domain.
To do item modeling, one constructs sets of
alternatives for different “facets” or variables.
For example:
Facet
One: Setting
a)
unscheduled visits
b)
scheduled appointments
c)
rounds
d)
emergencies
e)
other
Facet
Two: Physician Tasks
a)
history
b)
physical exam
c)
lab and diagnostic studies
d)
formulation of likely diagnoses
e)
selection of most likely diagnosis
Facet
Three: Case Cluster
1a)
initial workup, new patient
1b)
initial workup, known patient
2a)
continued care of known patient, old problem
2b)
continued care of known patient, worsening problem
3)
emergency
Once
the facet sets are created, then many items can be
generated by “rotating” the factors.
 |
Submit
a Summary! |
 |
|
If you've written
summaries or reviews of books on
teaching and learning, we'll include
them here and credit you. You
can email them to us at cogsim@cogsim.com.
|
|
|
©
1999-2001, CogSim
