Imagine you invested in a really expensive course to improve your presentation skills. But how do you tell if it truly made a difference? That's the challenge at the heart of measuring L&D effectiveness.
In this article, we’ll explore some of the most fundamental things you need to know about measuring Learning and Development.
Table of Contents:
What is learning evaluation?
"Evaluation is the total worth or value of L&D in monetary and non-monetary terms" (Sandler-Smith, 2006). So, really, there are 3 key elements here: worth or value, monetary and non-monetary. Let’s try to make sense of these.
Worth or Value of L&D
What exactly is the worth or value of L&D? We provide trainings and coaching, mentoring, on-the-job learning, online courses and many other learning experiences.
But why is that important? You could say that we help people develop and therefore, the organization develops through its people. And that make sense. But let’s be honest – no Chief Financial Officer will ever be satisfied with such an answer. What they need to know is what is our worth in numbers. And so we arrive at the other two key elements – monetary and non-monetary value.
Monetary L&D metrics
In the monetary metrics, you would look for things like:
The L&D budget and spending
How many people attended
How many didn’t show up
How many new learning programs were launched
How many training hours each employee had
Return on investment
Cost of training vs. cost of not training
Reduction in turnover costs
Increased sales or production output
Reduction in waste or error rates, etc.
Non-monetary L&D metrics
In the non-monetary column, you would put things like:
How satisfied were learners after each intervention
What was the employee engagement with various L&D activities
How much time does it take to master a competence
How was performance improved as a result of the learning
How confident learners were before and after the learning event(s)
How many internal promotions happened
What was the employees turnover rate
How learning events have influenced the organizational culture, etc.
What to measure
There are several authors and popular models out there (like Kirkpatrick’s 4 Levels and Phillips ROI). But at the heart of it, there are 7 things you want to measure:
Reactions – Did participants like the learning event(s)?
Knowledge – What knowledge did participants gain and retain?
Skills – What skills did participants gain and retain?
Attitude – How are participants behaving differently?
Application – Are participants applying what they have learned back on the job?
Results – What is the effect of the learning event(s) on the business’s bottom line?
Return on Investment (ROI) – what is the monetary return on investment of the learning event(s)?
How to measure it
At your disposal, you have several research methods and data collection options. Let’s break it down for each of the 7 metrics:
Reactions
Who? Participants/Learners
When? End of learning event
How? Happy sheet/satisfaction survey
Knowledge
Who? Participants, learning facilitator
When? Before, during and after learning event
How? Pre-course test / post-course test
Skills
Who? Participants, learning facilitator, L&D staff
When? Before, during and after learning event
How? Role plays, case studies, exercise, interview with participant’s manager(s)
Attitude
Who? Participants, their supervisor(s), reports and/or peers
When? 3-6 months after learning event
How? Survey (360-degree feedback), interviews with participants’ and their supervisor(s), observations, performance appraisals
Application
Who? Participants, their supervisor(s)
When? During and 6-12 months after learning event
How? Action plans created during learning event(s), post-event interview of participants and their supervisor(s), questionnaire
Results
Who? Participants and control group
When? 6-12 months after learning event
How? Cost-benefit analysis, tracking operational and performance data, employee retention, sales revenues, engagement surveys, number of customer complaints, customer services ratings, operating costs, product defects, etc.
ROI
Who? L&D Staff
When? 3 to 12 months, end of the fiscal year
How? Cost-benefit analysis, trend lines, participants and supervisor’s estimates, ROI formula.
Challenges of measuring L&D
If I speak Spanish fluently, is it because of my teacher, the curriculum, the classmates who helped me, or because I lived in Spain for a year where I had to speak Spanish?
This is the biggest challenge of measuring L&D. There are many different factors affecting someone’s performance. How can you isolate the ones that are directly affected by learning?
There isn’t a simple way to solve this. As an L&D professional, you must employ a nuanced approach, combining quantitative metrics with qualitative insights to capture the holistic impact of learning experiences. This involves not just looking at test scores or completion rates but also gathering feedback, observing behavioral changes, and considering external factors that contribute to performance.
Sources:
Elkeles, T., Phillips, J. and Phillips, P. (2017). The Chief Talent Officer. 1st ed. Abingdon, Oxon: Routledge.
Kirkpatrick, D. (2007). The Four Levels of Evaluation. Alexandria: American Society for Training & Development.
Phillips, J. J. (1991). Handbook of training evaluation and measurement methods. (2nd ed.). Houston, TX: Gulf.
Sadler-Smith, E. (2006). Learning and development for managers. Cornwall: Blackwell Publishing.
Commenti