Today's Tip
Measuring Transfer for Results and Glory
 DSA
would like to thank Dr. Jeanne Farrington for contributing this month’s
tip: Measuring Transfer for Results and Glory.
How
are your courses doing? More to the point, are your learners using
what they learned? In other words, what kind of Level III (transfer)
results are you getting?
 When we ask if conducting transfer evaluations is important, most
training professionals say “Yes!” Training professionals and management
teams generally want to know whether their leaders, managers, employees,
customers, and partners are using what they learned.
However, when we ask what training professionals are doing to measure
transfer, they often say, “Well, we’d love to do that,
but even if we were sure how to do it, where would we find the time?”
Here’s
an easy six-step method for not only measuring transfer easily and
quickly, but also for using the results to market your courses
to potential learners (and to justify your existence to an upper
management team that is always looking for ways to trim the budget).
- Decide
who will work on the evaluation. You can use a training coordinator
to set things up (1 day or less), but you should
use an independent
instructional designer (one without a vested interest
in the outcome) to write the questions, conduct interviews, and
write
up the results
(2 or 3 days).
- Make some decisions
about which learners to include in your evaluation.
a. |
If
you tested the learners at the end of the course to see who
mastered the objectives, choose only those learners who passed
the test. |
| b. |
Include
learners who took the course recently enough to remember
it and long enough ago to have had time to use what they
learned. |
| c. |
Decide
how many learners to interview. For a recent award-winning
project, we randomly selected 10 learners who met the criteria
for selection. |
- Choose your
learners and request their participation.
a. |
Generate
a list of everyone who took the course during the selected
time period and who also passed
the test (assuming
there was one). |
| b. |
Number
the learners and select a pool of perhaps 50 learners using
a random number table. (Search for one on Google if you don’t
have one handy. Often they appear with instructions about
how to use them.) |
| c. |
Write
an email message or a telephone script and use it to invite
your selected learners to participate. Tell them that you
want to provide your learners with the best quality courses
and that they can help. Ask if they’ve had the opportunity
to [overall course topic here] since they took the course.
If the answer is “yes,” then ask them if they
are willing to participate in a 15-minute telephone interview. |
| d. |
Schedule
the interviews with the first 10 (if that’s your chosen
number of participants) who say “yes.” |
- Write your
interview questions. Ask everyone the same questions. Take excellent
notes and/or record
the calls.
(You can use
a survey, but it’s harder to get good quotes.)
a.
|
Ask
a general question about whether they’ve had a chance
to [overall course topic here] since they took the course.
You’ll find out if the course topic is something they
do regularly or once in awhile. |
| b. |
Ask questions
based on the course objectives. Ask what tools, concepts,
or skills they’ve used to do [Objective 1, 2, 3, etc.]
since they took the course. Use the key objectives. Ask about
them one at a time. You might also include an enabling objective
or two here, if it’s important for you to note their
use of a particular skill. |
| c. |
Include
a “catch-all” question at the end. Something
like, “Is there anything I haven’t asked that
I should have asked about what you’ve used on the job
from the course?” |
- Write up
your results. Create a document that describes your study. Include
a summary of the
participant’s
answers.
a.
|
Overall,
are they using what they learned on the job? |
| b. |
Objective
by objective, how many are using, not using, or perhaps planning
to use what they learned for each one? |
| c. |
For each
objective, include helpful quotes from the participants about
how they are using what they learned on the job. |
- Do something
useful with the information you have collected. For example:
a.
|
Use
positive results and quotes in announcements or marketing
materials for your course. |
| b. |
Create
a summary of positive results and report them to your management
team so that they will realize what great work your department
is doing. |
| c. |
If there’s
some part of the course that no one is using, figure out
why and decide whether to discontinue it, modify it, or try
to change something in the environment so that people will
use it. |
Conducting
transfer evaluations can be done fairly quickly. The benefits include:
- Improving
results by making changes based on what does not transfer
- Marketing
your courses by sharing positive results and quotes with potential
learners
- Impressing your
management team with the wisdom of continuing to fund
your department
Not bad for
a few days’ work.
Until next time,
Darryl
PS.
Jeanne Farrington, CPT, Ed.D., is President of J. Farrington
Consulting and President-Elect of the International Society for Performance
Improvement. She is presenting a full session on transfer evaluation
at the 2007 Learning
and Performance Conference in June, with how-to
tips and timesavers from her experiences.
Article © 2007
Darryl L. Sink & Associates, Inc.
|
Vol.
3, No. 3 February
13,
2007 
