This is the report from the online workshop held on Tuesday 13 March 2001 on the subject of evaluation.

Stephanie Phillips Hi Sue, thanks for coming along today. I'm just going to wait a few minutes before starting the session to see who else turns up.

Stephanie Phillips Hi Stewart - welcome. Just waiting a few minutes to see who else is coming along.

Stephanie Phillips Hi Robin, thanks for coming.

Robin Cox Hi Stephanie. Looking forward to it.

Stephanie Phillips While we're waiting to see whether there are any others coming along, can we start with quick introductions - who you are, what you'd like this session to cover.

Robin Cox I'm an e-learning designer and coach, previously into IIP in a big way in the software industry. And conscious of some of the difficulties!

Sue Simpson I'm T&D manager at the University of Birmingham looking after support staff. Have been involved with IiP with 3 companies would like to explore how other people evaluate learning within their organisations.

Stewart Desson Hi, I'm a self-employed trainer and consultant. I'm interested in any measurement of changing behaviour through interventions e.g. self-esteem or time management training. I believe in getting quality feedback at the end of a course, but want ideas on avoiding the "happy sheets" scenario!

Stephanie Phillips Thanks all. I was a training officer in a previous life before I came to TrainingZONE. What got me thinking about this topic was an interview I did recently with a soft skills trainer, who basically called into question the whole idea of evaluating it in terms of ROI. It got me thinking about all the evaluation procedures I'd used in the past, and whether striving to 'do evaluation' properly was really worth the effort! IiP seems to be given more and more importance these days...

Robin Cox Fortunately IiP has become more practical than it was, Stephanie.

Stephanie Phillips Yes, they reviewed the standard fairly recently, I recall.

Sue Simpson I think evaluation is always worth looking at its how much effort you put into it. What is the value of quantitative and qualitative evaluation. Some of the best evaluation tools for me have been anecdotal but is this an acceptable evaluation tool?

Stephanie Phillips Sue - I think so, but again, I guess you need to write it down so it's recorded. Working on the basis that some evaluation is worth doing for itself then, what works best? It seems that the whole administration process is a key stumbling block...

Sue Simpson The new IiP standard as I understand it does not require 'written evidence' at individual level only that 'people understand the benefit of T&D'.

Robin Cox I think that's because it's all too easy to invent a form rather than to ask what information we really need. I know that I've fallen into this trap before.

Sue Simpson How this is then managed is the difficult one for me.

Stephanie Phillips Sue - it sounds like the written evidence presented too much of a stumbling block for people working towards the standard. I agree with Robin that you need to establish what you're evaluating first, and probably before the training takes place. Does anyone use pre-and post-evaluation?

Sue Simpson Is this because it only provided evidence of what had been achieved or not but not enhance what had been achieved or not?

Stephanie Phillips Sue - can you clarify, I'm not clear what you mean - thanks!

Sue Simpson Yes, I use pre and post course evaluation and where possible off job training is accompanied by a post course project. I think what I was trying to say is evaluation just a paper chase exercise which does not improve learning.

Stephanie Phillips Does anyone else want to comment? This is your workshop, after all!

Robin Cox But Sue, evaluation is not intended to be a paper chase. How did you show the connection between the learning and its benefits?

Sue Simpson It depends at what level , individual, team or organisation. I find evaluation at individual level hardest, getting individuals to see the need for evaluating the process for themselves.

Stephanie Phillips The trick is to keep the paperwork to the minimum while collecting as much information as possible, isn't it? Sue - definitely agree! I think this is partly due to the historical emphasis on being 'sent on a course' - it's out of your control, you don't have a vested interest in it.

Sue Simpson How do others encourage self evaluation?

Stephanie Phillips Stewart, Jeanette, any thoughts?

Brenda McNamara We use PDPs for self evaluation and to establish training needs.

Stephanie Phillips Thanks Brenda - how does that work?

Brenda McNamara We have a form that everyone completes twice a year and then you sit with your line manager to see how well you have done or not done as the case may be and you look at what could be done better and how. This could then throw up specific training needs.

Stephanie Phillips How does the training itself get evaluated?

Brenda McNamara That's the problem - we have a specific form which is an absolute pain to complete and send to each individual - I'm trying to establish a better way to go forward with this. Mostly it is done verbally at the moment and by seeing improvement on the job. Our IiP assessor was very pleased with the way we did it, but I could do with more formal records so that these can be analysed and we could learn the cost benefits too.

Stephanie Phillips Does anyone have any suggestions for Brenda? We need to get a discussion going here if we're going to continue this session, otherwise it's unfair on those who are contributing. Brenda - there's an electronic system which does the admin for you which will shortly be appearing on TrainingZONE - it might just help. I'll forward details to you if you're interested.

Brenda McNamara That sounds like a good idea - yes I am interested.

Stephanie Phillips Okay - I'll forward the information after this.

Brenda McNamara What do other people do about evaluation, cost analysis etc?

Robin Cox In my experience the difficulty is getting busy people to see any value in writing things down. So perhaps another way might be to use a tape recorder and interview them. Then you do the writing.

Stephanie Phillips Robin - nice idea! I guess all you have to do is pin them down for the interview :-)

Robin Cox This is true!

Stephanie Phillips Okay, I think we'll wind things up here, as there seems to be a lack of enthusiasm for a proper discussion. Thanks to Robin, Sue and Brenda for taking part. Hope to see you all at another session soon.

Robin Cox Thanks Stephanie. It's a tough subject.

Stephanie Phillips Robin - agree, there's no quick fix here!

Brenda McNamara Thanks Stephanie and I'll look forward to receiving the info.

Stewart Desson Thank you.

Stephanie Phillips Yes - I'll e-mail that through to you.

Stephanie Phillips Okay, I'll finish the session here. If anyone does want to stay behind and discuss, happy for people to stay logged on and chat.

Stephanie Phillips Thanks all and bye.

Brenda McNamara Bye for now - gotta go.