Demonstrating Return on Expectation

Why is this important?

How do you know whether the training and coaching you have invested so much in is having the desired impact? How much of your training budget is wasted on people who do not need, want or value it, are not actually ready for it, do not do anything with it? Training suffers from a similar reputation-damaging malaise as marketing: we know it is key to the success of our business, we know that we are getting a return on some of the money we invest; we just don’t know which part of the investment or how much of a return.

Most learning professionals are familiar with Kirkpatrick’s Four Levels of Training Evaluation:

  • Relatively few, however, know how to effectively get beyond Level 2

  • Most training courses end with the so called ‘happy sheet’ (Level 1), some include pre- and post-tests (Level 2) and, paraphrasing the Kirkpatrick Foundation, the rest (Levels 3 & 4), is left to hope; hope that the participants will do something with the learning; hope that their managers will support them and hold them accountable; hope that the targeted outcomes will show in the business results – assuming that these have been identified in the first place!

In his book, Evaluating Training Programs: The Four Levels (1st Edition, Berrett-Koehler, 1993), Don Kirkpatrick wrote: ‘Trainers must begin with desired results and then determine what behavior is needed to accomplish them. Then trainers must determine the attitudes, knowledge, and skills that are necessary to bring about the desired behavior(s). The final challenge is to present the training program in a way that enables the participants not only to learn what they need to know but also to react favorably to the program.’

In other words, in order to build an effective and impactful course or programme and evaluation methodology, we need to begin with the end in mind: working backwards through Levels 4-1, and then, collecting data forwards (levels 1-4) to build a ‘compelling chain of evidence’ that demonstrates the value of the training and delivers a tangible return on expectation. As budgets become tighter and spotlights more focused, the need has never been greater to be able to show how the training budget is contributing to employee engagement, productivity and ultimately the organisation’s goals.

What do we mean by return on expectation?

It is not RoI. We meet many clients who ask, ‘how can I measure RoI?’ in relation to learning and development. Return on Investment has become part of everyday business vernacular to describe value, which is to misunderstand that RoI is a financial metric. ROI is defined as the ratio of money gained or lost on an investment, relative to the amount of money invested. To isolate the impact of training in this way is a challenging and costly undertaking in itself, especially when one considers that there are many other factors, critical to realising value from the training, which occur outside the training environment, before and after delivery. Measuring return in purely financial terms is narrow at best, and risks devaluing the rich returns that can be witnessed in employee engagement, goodwill, creativity, discretionary effort and organisational culture, to name but a few areas.

RoE begins with the desired organisational outcomes, clarified and refined based on the expectations of key stakeholders and distilled into achievable, observable and measurable Level 4 results, which become the targets against which the success or otherwise of the training is measured.

In order to deliver the desired RoE we need to identify the required Level 3 behaviours and crucially the organisational drivers that support application and accountability that, if consistently performed, will most likely deliver the Level 4 results. Then we need to identify the Level 2 knowledge, skills and attitude that will enable and support these behaviours and finally, translate that into a relevant, engaging and effective learning experience (Level 1).

RoE is demonstrated by measuring and gathering evidence against the agreed success criteria at each of the four levels, starting with Level 1.

How do we deliver and demonstrate a return on expectation?

We subscribe to the spirit of the Kirkpatrick Foundational Principles:

1. The end is the beginning

2. RoE is the ultimate indicator of value

3. Business partnership is necessary to bring about positive RoE

4. Value must be created before it can be demonstrated

5. A compelling chain of evidence demonstrates value

1. The end is the beginning

We pride ourselves on starting every project with a blank sheet of paper and an enquiring mind. From the smallest to the largest project we seek answers to a number of key questions, such as:

  • Why are you doing this?

  • What are the organisational drivers or needs?

  • How will the organisation be different as a result?

2. RoE is the ultimate indicator of value

Depending on the size of the project, we engage with or at least seek to understand the expectations of the key sponsors and stakeholders by asking:

  • What are your expectations?

  • What does success look like for you?

From this we can build a clear and realistic picture of the expected outcomes that can be translated into achievable, observable and measurable Level 4 results. On occasion we might be drawn, even invited to encourage the sponsor to ‘think bigger’ or envision a broader expectation. Talking about RoE allows the conversation to stray way beyond the confines of financial or numerically measurable outcomes.

3. Business partnership is necessary to bring about positive RoE

According to Robert Brinkerhoff, training events alone typically result in only 15 percent transfer of learning to on-the-job behaviour. The largest RoE occurs through the identification and implementation of organisational drivers to support application and accountability of learning-on-the-job, including management support as well as processes and systems that reinforce, monitor, encourage or reward demonstration of the desired behaviours.

Again, depending on the size and scale of the project, we partner with L&D professionals, managers and participants to understand:
  • What are the critical behaviours that will most likely deliver the desired outcomes? What will the participants need in order to support application and accountability back in the workplace?

  • What is missing and how will it be provided?

  • What do you see as the key challenges?

  • What has been tried before?

  • What is working well/not so well?

  • What is the learning culture and how do people like to learn?

From this we can design and develop the learning intervention including the event/s, pre- and post-learning support, the programme metrics and the means of measurement.

4. Value must be created before it can be demonstrated

Value cannot be created without application. Our Management Learning System, built on the award winning CALF platform, supports delivery of courses and programmes with pre- and post-learning support, discussion forums, learning materials and resources, Level 1 feedback and action planning, Level 2 assessments, Level 3 learning application reports and assessments and Level 4 metrics.

Crucially it enables managers to engage with and input into the learning journeys of their people with review and feedback at every stage, supporting them in applying the new knowledge, skills and insights, and staying accountable to their learning journey. We encourage participants and managers to discuss and agree learning objectives, outcomes, action plans, learning application and impact. The Management Learning System enables them to have informed productive conversations with real accountability, thereby increasing the likelihood of successful and consistent application of skills and behaviours that will deliver the desired organisational results.

It also provides us and the learning sponsors with an early warning detection system to flag and address issues at every level and to put in mitigation measures to bring things back on track.

5. A compelling chain of evidence demonstrates value

Applying these foundational principles enables us to define and agree the expectations of stakeholders and the evidence to be collected at the beginning of the project, to collect data along the way through our Management Learning System and to build a compelling chain of evidence of RoE.

The chain of evidence includes both numeric data and the testimonials and stories of those who participated in the process. This combination of evidence shapes a story that appeals to and is understood by everyone to increase engagement and confidence in the courses and programmes. In addition, it encourages continuing application of behaviours back in the workplace that deliver desired business results.

Programme sponsors and stakeholders can be reassured and confident that they have invested wisely and achieved a positive return on expectation.