A discussion-driven e-mail newsletter for training managers
Wednesday, Oct. 27, 1999 Vol. 1, No. 32
THIS WEEK:
1. Why we can't count on Kirkpatrick ...
2. ... but should count on learners
3. Challenge fest! No-shows, budget and more
___________
DON'T COUNT ON KIRKPATRICK ...
When consultant Saul Carliner asserted that training guru
Donald Kirkpatrick's four-level approach to evaluating
training won't work online, it set off a flurry of debate.
Why shouldn't Kirkpatrick work online? Tech delivery is just
another medium, like a classroom, one Kirkpatrick backer
argued ("Why Not Kirkpatrick?," Oct. 6).
Consultant Carliner (SaulCarliner@worldnet.att.net), a
faculty member at Bentley College in Waltham, MA, stands by
his original take on Kirkpatrick's four-level evaluation of
training.
(The levels: 1-Did they like it? 2-Did they learn? 3-Did it
change their behavior? 4-Did it affect the bottom line?)
In contrast to classroom delivery, Carliner says, tech-
delivered content has five separate learning contexts:
o Collaboration
o Formal education
o Knowledge management
o Performance support
o Training
For detail on these contexts, Carliner suggests his OnLine
Learning White Paper at http://www.onlinelearning99.com.
The Kirkpatrick model doesn't transfer cleanly to all five
categories of tech-delivered training, says Carliner.
Kirkpatrick accentuates transfer of learning -- did they get
it?
This isn't the case with performance support. Rather than
trying to embed knowledge in workers' memory, performance
support tries to guide workers through a task, perhaps only a
very few times -- or perhaps just once.
Take the example of loading software. A computer user,
with the assistance of performance support -- online prompts
-- may only have to install software once. "Where's the
transfer there?" asks Carliner.
In such cases, an adapted version of the Kirkpatrick model
might work. Carliner offers his article, "Demonstrating the
Value of Technical Communication Products and Services," in
Technical Communication, third quarter 1997. It is at
http://web.bentley.edu/empl/c/scarliner/idbusiness/value.htm.
NOT WHAT TRAINERS WANT?
Another Kirkpatrick limitation is that it exclusively targets
"summative evaluation," says Carliner -- that is, Kirkpatrick
comes into play only after a course is in use.
But in online learning, pilot-testing of courses --
"formative evaluation" -- is critical for success, Carliner
avers.
Finally, Kirkpatrick primarily assesses outcomes that are
intended -- outcomes that trainers want learners to achieve.
But, argues Carliner: "In the learning contexts of
collaboration, knowledge management, and performance support,
many of the outcomes are not intended -- because they cannot
be anticipated."
A goal of knowledge management, for example, is that users
make their own connections among information in the knowledge
base.
"Certainly the Kirkpatrick model provides a useful starting
point," Carliner concludes. "But as Kirkpatrick anticipated,
it requires extension and adaptation."
TDF E-NET SAYS: Carliner's presentation Feb. 21 at TRAINING
2000 ( http://www.training2000.com) in Atlanta is called
"Future Travels of the InfoWrangler: Career Survival for
Trainers in the Era of Online Learning."
_____________
... BUT DO COUNT ON LEARNERS
Two readers add this to the evaluation debate: Pay attention
to what learners tell you.
How participants apply training back on the job, or how much
money training saves, are hard to measure even for classroom
training, says Pat Cunningham-Reid (p.cr@csuohio.edu),
training and development manager with Cleveland (OH) State
University.
So: "I have been using a method for several years now that,
although not perfect, is certainly better than nothing,"
Cunningham-Reid says. "I ask the participants for Level 3 and
4 evaluations as a post-class evaluation.
"It comes down to a matter of trusting that they know what
they are and should be doing. Empowerment means we trust 'em
with a huge part of our business -- so why not with their
concepts of course worth?"
'SHINY STONES'
In the same vein, this response:
We asked you to tell us your "evaluation mantra" after reader
Gwen Thomas (gthomas@ciber.com), a Lake Mary, FL, consultant,
volunteered hers: "Learn, remember, get better."
Dave Barnette (BarnetteDH@mail.ports.navy.mil), employee-
development specialist at Portsmouth (NH) Naval Shipyard,
says his mantra is: "Trust adult learners' evaluation of
necessary training" -- "TALENT," for acronym fanciers.
"With all respect to Don Kirkpatrick's four-level evaluation
theory," says Barnette, "my experience suggests that we
should LISTEN to what training participants say about which
parts of our training are necessary, and what we might do to
make the training better and more relevant to their work.
"Having listened, we should then adjust the training, ask
again, adjust again, and trust people to tell us when we get
it right.
"And shame on us if they ask for bread and we give them a
stone -- even a really shiny stone, or Seven Highly Effective
Stones."
_____________
CHALLENGE FEST! READERS ASK FOR IDEAS
Can you help? Send your response to tdfenet@lakewoodpub.com
Please include your name, title, organization, what it does,
where it is, and a phone number at which we can reach you.
Text only, please. No attachments. But include a Web-site
address, if you like, to which we can point readers.
Please use the given subject line (e.g., Phase It In?).
PHASE IT IN?
An airline trainer asked how much to pay for a training-
management system, and we've been offering your guidance
("Shop around: What price training-management systems?," Oct.
20).
Here's a related question:
"I also work for a large global airline and am currently
researching the options for training-management systems.
"Online learning is still largely in a development stage,
with new enhancements and capabilities happening everyday.
With this in mind, I am considering developing an in-house
solution in a staged approach, but utilising the expertise of
a supplier to do so.
"This seems to be a solution to our budget constraints. Is
anyone else approaching this in a similar way?"
HOW DO YOU GET THEM TO SHOW UP?
"My biggest challenge is getting employees to actually show
up for training that they are scheduled for. A 20% no-show
rate is not uncommon. Pleas to supervisors and managers have
fallen on deaf ears.
"I am now thinking since I can't get accountability, maybe I
should try another angle -- a reward system or some sort of
motivator.
"I work for a government agency, so we are somewhat limited
as to what we can do. I would like to know what others have
done successfully that doesn't cost money."
COURSE-CREATION SOFTWARE ON A BUDGET?
"What software is effective enough to help create and deliver
courses without depleting a small-business bank account? All
we've seen until now is out of our reach for our company
because of price."
TESTING FOR ENGINEERS?
"Do you have any ideas as to where I can find tests to give
engineering applicants as part of a recruitment process? This
would be to evaluate skill levels."
WHAT'S YOUR CHALLENGE DU JOUR?
Are you stuck? Maybe your peers have some ideas on how to get
you out of a tough training-management spot. E-mail
tdfenet@lakewoodpub.com with an account of your challenge.
We'll post it here, without your name, and invite readers to
offer their thoughts. Please use a distinctive subject line.
_____________
... AND FINALLY
*************************************************************
Up next: TRAINING 2000 with Nobelist Desmond Tutu, MIT's
Nicholas Negroponte, anthropologist Jennifer James, Fortune's
Tom Stewart and TV's Ben Stein: http://www.training2000.com.
*************************************************************
o It's free! It's weekly! To receive TDF E-Net, go to
http://www.lakewoodconferences.com/,click Request a
Brochure and fill out the form; or e-mail
lakewood2@list.emailpub.com with "TDF-E-NET SUBSCRIBE"
in the subject line.
o If you prefer NOT to receive this free weekly newsletter,
please UNSUBSCRIBE by responding to this message with
"unsubscribe TDF-E-NET" as the subject. Or e-mail
lakewood2@list.emailpub.com with "unsubscribe TDF E-NET"
as the subject.
____________
Copyright 1999
Bill Communications Inc. (Lakewood Publications Inc.)
Distributed by MessageMedia Inc. http://www.messagemedia.com
