Can you really measure training ROI?

Can you really measure training ROI?

Concern about the effectiveness of training programmes is boosting demand for training evaluation.

Satisfaction surveys, which are widely used to- day, are a vital part of instructional engineering and can prove invaluable if the right questions are asked.

However the emergence of new approaches and new tools suggests that training professionals may be able to go a step further to demonstrate the value of the programmes they offer.

At Cegos, we believe the questions that to be addressed are:

Can the new tools really measure the effectiveness of training programmes?

Is it strictly necessary to measure the return on investment (ROI) for training courses?

Can data prove ROI?

Today, there are increasingly efficient tools for measuring the time spent and the attention given to online training programmes. Learning analytics can capture data about course participants’ activities in relation to the programme, and observe their interactions with each other.

Automatically calculating interactions makes it possible, for example, to measure the number or frequency of online users’ connections to the training portal, the type of documents they shared, the average duration of their visits or of the interactions among participants.

Focus your efforts on the success factors

It can sometimes be relatively simple to assess a training programme’s ROI. For example, if you were to conduct a training course for salespeople in a retail phone store. You might start by comparing the performances of two groups of salespeople, one of which had been trained and the other not, we could then begin to identify the impact of the training on the sales figures.

But, even if we know how to assess ROI, we also know that an accurate assessment often costs far more than the training programme itself.

Measuring the ROI for a company starts with asking:

“What will happen if we don’t do anything?”

This means weighing up the risks of not doing any training, such as: a lack of understanding of corporate strategy, inertia, resistance, individualism, apathy, staff turnover, etc.

Moreover, it is often more cost-effective to focus on the success factors for a training course rather than on its evaluation afterwards:

It helps to focus on a few key points:

• Prepare beforehand: introductory presentation, seminar, launch communication, teasers, etc.
• Get the learners involved: get them to think about what they stand to gain from the training, the objectives they can set themselves, help them if necessary to compile an application for certification, etc.
• When the time comes to evaluate the course, ask the learners what they got out of the course, about their level of confidence and engagement about putting what they learnt into practice.
• Use questions such as: “How confident are you about applying these new tools on the job?”...
• Set up post-course support and guidance in the workplace by making it easy for the frontline manager, an in-house or outside tutor to get actively involved.

From ROI to ROE

There is an alternative to measuring ROI, which is to measure ROE, the Return On Expectations.

This approach, widely publicised by James and Wendy Kirkpatrick in a 2010 article, shifts the emphasis to achievement of the operational objectives.

ROE can be evaluated on four levels:

1. The ‘expectations’, defined well before the training project, are discussed by all of the stakeholders, from the training manager up to senior management. It is an opportunity to check whether the course was indeed the right solution to meet the expressed needs.
2. The expectations are then translated into measurable results, expressed in clear, general terms. The question asked at this level is: “what results will tell us that the course is a success?
3. These measurable results are then used to identify - and ultimately achieve - the critical behaviours expected on the job.
4. Lastly, the behavioural training objectives are defined, in direct relation to the targeted behaviours (they are therefore expressed in such a way as to be observable and measurable on the job).

These objectives will serve as a basis for designing and delivering the training.

The value of the training is therefore based on concrete proofs, evaluated on the job and in relation to professional behavioural objectives.

Moreover, by getting all of the stakeholders involved beforehand, the ROE approach is an excellent way to elicit their support for the training project. This, in itself, is a powerful factor in its effectiveness.