8 Responses

1. how long is a piece of string?

   Hi Sarah

   I would doubt that such a thing exists; it is too reliant on the initial learning objectives that you set for the intervention.  I've seen level 2 evaluations 'done':-

   -as a yes or no answer to the question "Did the training fulfil the stated objectives?"

   -as a sliding scale 1 to 5 (one is barely and 5 is completely) to the question "To what extent did the training fulfil the stated objectives)

   -as either of the above for "your personal objectives" rather than the "stated objectives"

   -by the trainer/an assessor in the form of observations of peoples' comparative success in role plays……

   -and practical demonstrations.

   -as multiple choice quizzes

   -as simple written tests

   -as more complex essay tests.

   Sorry this doesn't answer your question but I hope it helps.

   Rus Slater 

    

    

2. Agree with Rus…

   Assuming that you are referring to the New World Kirkpatrick model; level 2 i s learning and is defined as; 'to what degree participants acquire the intended knowledge, skills and attitudes based on their participation in the learning event.'

   So as Rus says, it would be difficult to find a universal benchmark as it is wholly linked to the learning outcomes. Agree with Rus' list of how it might be done but also don't forget to involve the operation in evaluating this.  So, for example, if it's sales training and requires an attitudinal change in approaching customers; you may get the learners managers to observe this for you, (over a period of time), to see if the change in attitude has been embedded consistently.

   Don't forget that level 1 is purely the reaction to the learning event so level 2 has to be more than a happy sheet, it has to be a clear demonstration that learning has occured (aligned to outcomes).

   Hope that helps

3. More Detail

   OK, to expand!  Here's a quick learning journey outline:

   TNA (100 questions and Psychometric  evaluation)

   Pre course:

   Line manager scores delegate against a competency framework

   50 question knowledge assessment

   SWOT analysis 

   Learning Intervention: 3 Day Course – including skill observation

   Action planning

   Post Course

   Repeat of 50 question knowledge assessment…..with a benchmark of 100%

   Webinar that review the success of action plan

   Repeat of line manager competency scoring 

   My problem is with the repeat of the 50 questions post learning.  I think that's too many and 100% is to high a benchmark.

    

4. More Detail

   OK, to expand!  Here's a quick learning journey outline:

   TNA (100 questions and Psychometric  evaluation)

   Pre course:

   Line manager scores delegate against a competency framework

   50 question knowledge assessment

   SWOT analysis 

   Learning Intervention: 3 Day Course – including skill observation

   Action planning

   Post Course

   Repeat of 50 question knowledge assessment…..with a benchmark of 100%

   Webinar that review the success of action plan

   Repeat of line manager competency scoring 

   My problem is with the repeat of the 50 questions post learning.  I think that's too many and 100% is to high a benchmark.

    

5. More Detail

   OK, to expand!  Here's a quick learning journey outline:

   TNA (100 questions and Psychometric  evaluation)

   Pre course:

   Line manager scores delegate against a competency framework

   50 question knowledge assessment

   SWOT analysis 

   Learning Intervention: 3 Day Course – including skill observation

   Action planning

   Post Course

   Repeat of 50 question knowledge assessment…..with a benchmark of 100%

   Webinar that review the success of action plan

   Repeat of line manager competency scoring 

   My problem is with the repeat of the 50 questions post learning.  I think that's too many and 100% is to high a benchmark.

    

6. Too vague still…

   Hi there, it seems very comprehensive but still not sure specifically what it is they will  learning, therefore what they will be doing differently as a result of the course and even more importantly what specific business measure  it will impact upon. 

   These measures may or may not be right.  For example, if it is a course about how to deal with dangerous chemicals beacuse the nature of the business needs everyone to be absolutely compliant it may require a 100% benchmark.  It's all about the specific skills they are learning and how important it is to drive specific business measures the new learning will impact upon.  Hope that helps

7. as clive says…..

   …..it does depend of the nature of the content and of the questions they have to answer in the knowledge assessment (before and after).

   With regard to 50 being too many, the challenge is balance; if the delegates (or their line managers think 50 is too many (because of time) then you may have a political problem on your hands.  Too few questions to actually assess that they have learned anything (enough) is no good either.

   Perhaps you need to defer to the client about the "pass rate"……and even that will depend on the format of the questions; multiple choice should probably be a higher pass rate than free field for example.

   Can you share more detail or is that potentially confidential?

   Rus Slater

    

8. Another thought…

   Not sure if you are an internal worker or a freelancer (so there may be a cost implication) but would it be possible to pilot the approach with a smaller group to test out whether it works.  You could start creating your own benchmark then.

   So, if the pilot group go through it an none of them achieve 100% you know there is some work to do. If only a small proportion don't pass there may be some smaller tweaks to make.

   I still think the validity of the approach is related to how important the learning is to the organisation.  Another example I remember when I was going through the Kirkpatrick certification process is that the US immigration service insisted on a 100% benchmark becuse the learning event was around when to deploy a firearm or not so it had to be absolutely right.