Today's Tip
Criterion-Referenced Checklists: The Workhorse of Evaluation
 DSA
would like to thank Dr. Barbara Martin for
contributing this tip on Criterion-Referenced Checklists.
Dr. Martin will be teaching more about this subject at DSA’s
The Criterion Referenced
Testing Workshop in San Francisco, September 25-26.
What if I told you there was an instructional design (ID) tool that
was so versatile that it could be used, with minor modifications, throughout
the entire ID process to create good instruction? You’d use it,
right? It exists and is a criterion-referenced checklist.
Look below at the portion of the checklist for creating good multiple-choice
test items.
| Portion of a checklist for a multiple choice test items |
Yes |
No |
1. |
Does
the stem include a complete thought? |
|
|
2. |
Is the item free
of cues? |
|
|
3. |
Are all the response
choices plausible? |
|
|
4. |
Are the distractors
arranged in a logical order? |
|
|
5. |
Are all response
choices of the same approximate length? |
|
|
6. |
Is there only one
correct answer? |
|
|
The obvious and
most likely use of this checklist is to evaluate whether or not learners
have developed multiple choice test items that meet the
quality standards specified in the checklist. However, having a checklist
allows you to accomplish many other aspects of the ID process. In the
example above if you consider that “creating multiple choice test
items” is
the final or terminal objective, here are some ways you can adapt the
checklist:
- The
skills stated in the checklist can be used to develop a needs
assessment survey since the listed
skills are prerequisites to the final
objective.
- The elements in a
checklist become the enabling skills for the final/terminal skill so
in essence you have begun to develop a task
analysis. From these,
you can write learning objectives too.
- The checklist can
also be used to guide the instructional process. Each skill listed must
be taught
in order for the learner to become proficient.
Use the checklist to make sure nothing is left out during instruction.
- The learner
can use the checklist as a self-check or even as a job
aid during instruction.
- When the learner is
back on-the-job, the checklist can be used as a prompt for doing the
job correctly, it can be used
to evaluate transfer
of training when used by a supervisor or manager, and it can be used to determine
what,
if any, skills need to be included in remediation training.
There are basically two categories of skills that require checklists. These
skills always have multiple components.
- Skills that require the
learner to produce a tangible object or project with multiple parts require
a product checklist. Examples of objects
or products that you might ask learners to produce include a blueprint, a
workplace action plan, an instructional unit or test, or a device or gadget.
- Skills that require
the learner to demonstrate actions require a performance checklists. Examples
of actions include making a telephone sales
call, performing CPR, making chocolate chip cookies, and teaching a class.
In order to design a checklist
the instructional designer must make a list of all the components of a
task, put the items in a logical order, and include
an evaluation scale to measure the adequacy of the learner’s response.
One of the easiest ways to make a checklist is to start with a product or
performance that you have already evaluated as excellent or acceptable and
work from it.
There you have the checklist – an ID workhorse – and one of
the most versatile tools in the instructional designer’s repertoire!
Once you start using them you’ll wonder how you ever got along without
it. You will find other uses for checklists too. Plus learners appreciate
the organization and structure that a checklist brings. Try writing one for
your next instructional unit, then sit back and revel in the benefits.
Until next time, Darryl
 P.S. Dr.
Barbara Martin is an active DSA associate and teaches The Instructional
Developer Workshop, The Course Developer Workshop, and The Criterion
Referenced Testing Workshop. She has written many articles and an award
winning book on the designing instruction for affective behaviors.
Article © 2007 Darryl
L. Sink & Associates, Inc.
|
Vol. 3, No. 9 July 12, 2007 
