Employing a Learning Measurement Strategy to Build a Measurement & Evaluation Ecosystem
As experienced learning measurement and evaluation practitioners, we have enabled dozens of clients to successfully implement our company’s learning analytics technology platform. While most clients experienced immediate improvements in their measurement maturity, some stalled in their attempts to build a culture that used data for improvement and decision-making.
We found that organizations who took a tactical approach to learning measurement implemented technology to gather and report on learning effectiveness but failed to address broader organizational requirements to build sustainable evaluation capacity. Conversely, successful clients applied a holistic approach by augmenting technology with measurement governance, processes, accountabilities, competencies and evaluation use.
This presentation to the American Evaluation Association Conference in 2012 depicts how we applied a systemic approach to build a sustainable measurement and evaluation ecosystem within our clients’ organization. We highlight methods to diagnose measurement maturity and enable clients to realize a culture of evaluative thinking that uses data for continuous improvement.
How We Guide Our Clients to Design with the End in Mind
All organizations need scalable and repeatable processes for measuring their programs. Strategic and costly programs require special treatment. These highly visible programs are under greater scrutiny, to not only ensure a high quality learning experience, but also equally importantly, to ensure that they make a difference, that is, impact a business outcome. The challenge for many of our clients is that they don’t sufficiently clarify the success indicators at the outset or the chain of influences between a set of inputs (resources), to short term impacts (improved job performance), to the ultimate goal of improving for example, employee engagement, cycle time, customer satisfaction or sales growth.
As a consultant, I have two goals when guiding clients through the process of designing with the end in mind:
- To make explicit the linkage between program inputs (resources, funding and technology), program activities (assessments, training, collateral and coaching), outputs, short-term results and ultimate outcomes.
- To use the exercise to demonstrate the importantce of systematically embedding this upfront process into an end-to-end training and development process.
This Book Chapter in “The Field Guide to the 6Ds” describes this process and its benefits to clients.