Training: Three Steps to Effectiveness

Training: Jim and Wendy Kirkpatrick –  11/4/10, Kirpatrick Partners, via clomedia.com

The front-page headline of USA Today on Feb. 16, 2010, read, “Homeland Security Reports Lost Guns: Some ended up with criminals.” In the first paragraph, there was this note: “The nation’s Homeland Security officers lost nearly 200 guns in bowling alleys, public restrooms, unlocked cars and other unsecure areas, with some ending up in the hands of felons. The problem, outlined in a new federal report, has prompted disciplinary action and extra training.”

This article shows how much of the world sees formal training as having a key role in both the problems and solutions of government and business. Poor economic conditions over the past several years have exposed common training practices as ineffective in achieving organizational goals. But common sense would indicate that lack of training is not the reason 200 guns were lost, nor is it likely the best solution to keep more from disappearing. Can you image the Department of Homeland Security simulating a restroom and having agents practice picking up guns from the sinks before leaving?

A shift from effective training to training effectiveness will produce the results that organizations want and need. Effective training is a measure of whether or not a training event met its objectives. Training effectiveness is a measure of whether training and all subsequent reinforcement helped workers perform specific tasks that contribute to organizational objectives. What this means is that the focus should move away from training events and isolated incidences of formal learning to a more holistic learning and performance support process. This effectiveness approach integrates learning and reinforcement into the work environment when and where they are needed to support the accomplishment of high-level organizational goals.

There are three strategic steps organizations can take to move toward training effectiveness:

1. Focus on business results.

2. Realign resources to support on-the-job learning and performance.

3. Make formal learning efficient and tactical.

Step 1: Focus on business results.

Any successful initiative, training related or not, starts with clear definition of the desired outcomes. When this is done properly, execution and demonstration of organizational value become quite straightforward. This can be accomplished by using the Kirkpatrick Four Levels model (Figure 1) in reverse during the planning phase: starting with Level 4 results, then moving backward to Level 3 behavior, then considering levels 2 and 1.

Once the plan has been created, the four levels are then used in numeric order to execute the initiative, measure the results and demonstrate the value. Using the four levels in reverse during the planning phase eliminates two common problems found with many initiatives: never getting to levels 3 and 4 and a focus on learning objectives instead of strategic goals.

Many hardworking professionals start initiatives with Level 1. Current statistics back this up: 78 percent of all programs are measured at Level 1; 49 percent at Level 2; 25 percent at Level 3; and 7 percent at Level 4, according to an ASTD Value of Evaluation study in 2009. These figures illustrate the first problem: When there is not a plan that starts with the highest-level result, organizations spend most of their time, money and resources on the lower levels and seldom get to the higher, more important ones.

Another challenge that starting with the four levels in reverse can eliminate is myopically focusing on training as the solution to the problem. For example, if there is a training program designed to teach participants to enter purchase orders correctly, the overall goal of the program would likely be tied to an increase in sales or profitability. Making the goal to teach participants how to enter orders correctly diminishes the overall importance of the initiative and keeps it stuck at levels 1 and 2. Examples of appropriate organizational level results include increased revenue, cost savings, customer satisfaction, brand recognition, earnings per share and market share.

As basic and simple as starting with the desired end result in mind sounds, this first step is where many initiatives become misguided. Research by Jack and Patti Phillips of the ROI Institute revealed that while 96 percent of CEOs want training leaders to tie their efforts to meaningful business results, only 7 percent comply.

Frustration with the seeming inability to relate a single training class to a high-level organizational mission is common. Leading indicators help to bridge that gap by providing short-term observations and measurements that suggest that critical behaviors are on track to create a positive impact on the desired results.

Business results by their nature often take several quarters or even years to manifest. Leading indicators are needed because they show stakeholders, learning leaders and participants if the initiative is moving in the right direction. They also provide data to confirm if training and reinforcement are working or require modification.

Step 2: Realign resources to support on-the-job learning and performance.

Once a suitable organizational-level goal is determined, the next step is to identify the Level 3 behaviors. These are termed critical behaviors because they are the few specific actions that if performed consistently on the job will have the biggest impact on the desired results. Critical behaviors are supported by required drivers: processes and systems that reinforce, support, monitor and reward performance of the critical behaviors on the job.

Figure 2 visually depicts the relative importance of each of the levels. Level 3 is the most important in achieving training effectiveness and is therefore in a large bull’s-eye configuration. Organizations that put a strong emphasis on the reinforcement of training through accountability and support can expect as much as an 85 percent transfer of learning to behavior. Conversely, companies that rely primarily on training events alone achieve around a 15 percent application rate, according to research from Rob Brinkerhoff in 2006.

Required drivers (Figure 3) are the key to accomplishing learning transfer. They decrease the likelihood of people falling through the cracks or deliberately crawling through the cracks if they aren’t interested in performing the required behaviors. These reinforcing processes and systems cannot merely exist. They must be actively executed and monitored.

Common excuses for not evaluating Level 3 are that it’s too difficult or too expensive; it’s not the learning department’s job; or business managers won’t support it. But based on the variety of required drivers available, it is neither difficult nor expensive to execute a Level 3 support plan. Learning and business professionals should together review the learning transfer statistics and come to an agreement during the planning stage of an initiative to determine who will perform and measure each driver.

Step 3: Make formal learning efficient and tactical.

Today, almost 50 percent of the evaluation budget is spent measuring Level 1. Add to this the 23 percent spent on Level 2 testing, which has little correlation to learners’ ability to later perform the task on the job, according to the 2009 ASTD Value of Evaluation Study. The end result is a disproportionate share of resources spent in areas with little organizational value.

Focus learning on teaching participants what they need to know to confidently perform the desired on-the-job behaviors. Get rid of the rest. This tactical approach will leave the resources required to support Level 3.

Failure of graduates to perform the desired behaviors on the job is often inaccurately diagnosed as lack of skill, so they are sent back for more training. In reality, only about 10 percent of learning transfer failures are due to training; 70 percent of the time or more it’s something in the environment, according to a 2006 ASTD study.

Adding Confidence and Commitment to Level 2 Evaluation

Two new dimensions have been added to Level 2 to help to close the gap between learning and behavior: confidence and commitment.

Confidence is defined as: “I think I can do this on the job.” Commitment means: “I intend to do this on the job.” Addressing these issues during training brings learners closer to the desired on-the-job performance. It can proactively surface potential on-the-job application barriers so they can be resolved. Gathering evaluation data on confidence and commitment also provides the information that learning professionals need to diagnose the root cause if learning transfer is substandard.

Some of the most common causes of lack of confidence and commitment are lack of clear expectations, unsupportive or toxic supervisors, a culture of non-accountability, conflicting or confusing priorities and a lack of ongoing resources.

Methods to increase confidence and commitment:

• Include the higher-level purpose and goal of learning in communications before and in discussions during the event. Make sure everyone involved knows the importance of applying what they learn when they are back on the job.

• Allow enough time for skills practice during training.

• Near the end of a session, facilitate a discussion about what it will be like to apply the new skills on the job. If anticipated barriers are mentioned, work collaboratively on solutions.

• Include a question about confidence in the after-course evaluation form. For example, “To what degree are you confident that you will be able to apply what you learned?”

If an underlying pattern in responses points to shortcomings in job culture or environment, get them addressed quickly so the learning has a chance of succeeding.

Bringing Engagement and Relevance to Level 1

The current investment in gathering Level 1 reaction data is far greater than the importance this level dictates. Many reaction sheets include phrases that start with, “The facility was…” and “The facilitator was… .” These types of questions ask participants to give opinions about trainers and learning rather than how the event prepared them to perform a new task on the job. The words “engagement” and “relevance” have been added to the Level 1 definition to reflect the higher purposes of measurement at this stage.

Some learners are demotivated by the mindset that they are being sent to training by their supervisors. This creates a lack of self-responsibility that transcends the learning experience and follows them back on the job. Level 1 evaluation questions can set the stage for learner responsibility if they are phrased as learner-centric questions instead of trainer-centric.

Assuming a four-point scale from strongly agree to strongly disagree, learner-centered Level 1 evaluation questions could be:

• I was fully engaged during the workshop.

• The classroom was free of distractions so I could focus on the course.

• The knowledge of the facilitator added richness to my learning experience.

• I will be able to apply what I learned in class on the job.

By transforming Level 1 questions to be as learner-centered as possible, participants will find more value in their learning experiences. It will lead them to take both their learning experiences and Level 1 reaction sheets more seriously.

Relevance is also of vital importance for learning, application and ultimate value. When participants are clear about why they are learning the information and how they will be expected to apply it on the job, there is a much better chance of success at Level 3 and accomplishment of Level 4 results. Here lies an opportunity not only to find out useful information about relevance, but to weave in other evaluation methods besides a survey immediately following a program.

Methods to measure Level 1 relevance:

• Use formative methods to ensure that participants see the relevance of the material during training.

• Use questionnaires and individual and group structured interviews.

• Consider delayed administration, as participants moving into new situations cannot give informed opinions about relevance until they have a chance to apply the information.

• Query line managers, subject-matter experts and program observers in addition to participants.

There is an urgent need for learning to redefine its role and boundaries. L&D professionals must move away from merely providing effective training and concern themselves with delivering learning effectiveness. This means redeploying selected resources from formal training to supporting on-the-job learning and performance. Put your flag in the ground by challenging training traditions and start new initiatives with a focus on accomplishing business results.

Feds Pioneer Effectiveness Approach

The U.S. federal government is leading the way in delivering training effectiveness. In December 2009, the Office of Personnel Management (OPM), the human resources consulting arm of the federal government, took measures to align all federal agency training initiatives with their respective missions.

OPM document 5 CFR 410 stated that agencies must develop and implement a process to evaluate training and development programs’ impact in terms of:

• Learning.

• Employee performance.

• Work environment.

• Contribution to mission accomplishment.

Further regulations direct agencies to evaluate each program or plan established, operated or maintained with specific performance plans and strategic goals and modify programs or plans when needed to accomplish those goals.

Basically, all non-uniformed federal government agencies must align their training to Level 2 learning, Level 3 employee performance and work environment and Level 4 mission. It is no longer enough to collect four-level evaluation data. The data must be analyzed and used to make program modifications.

The Internal Revenue Service’s Large and Mid-Size Business division (LMSB) helps customers understand and meet their tax responsibilities and applies tax law. Rob LaVanway, manager of L&E International, and his team in the LMSB Learning and Education division revamped its new-hire training program with the intent of generating higher outcomes while reducing training expenses.

They first identified targeted Level 4 outcomes and the on-the-job behaviors that would most likely lead to those outcomes. The current new-hire training program was analyzed against the newly defined goals and found lacking. The entire learning and performance plan was then redeveloped to fill gaps and support critical behaviors and drive results.

The new blended learning and performance model includes more e-learning modules, work cases and on-the-job coaching and reinforcement. This approach contrasts with the previous program, which was heavy with theory, hypothetical cases and instructor-led training. Their work is being used to further the business partnership emphasis within the IRS and other parts of the federal government.

James D. Kirkpatrick and Wendy Kayser Kirkpatrick are with Kirkpatrick Partners. They can be reached at editor@clomedia.com.

http://clomedia.com/articles/view/three_steps_to_effectiveness/1

Related Training Information:

https://mcdonaldsalesandmarketing.biz/category/instructional-design/

https://mcdonaldsalesandmarketing.biz/category/learning-theories/

https://mcdonaldsalesandmarketing.biz/category/how-to-transfer-training-learning/

https://mcdonaldsalesandmarketing.biz/category/blended-learning/

https://mcdonaldsalesandmarketing.biz/category/personal-learning-environments-2/

https://mcdonaldsalesandmarketing.biz/personal-learning-environments-ples/

==================================================================

Learning Optimized, Behavior Changed, Performance Advanced

My deliverables to you are: (I) Improved Learning Outcomes and Increased Competence; a. Retention to fluency (95% vs. 28%), b. Behavior change through accountable reinforcement, c. Improved application, d. Advanced individual performance and e. Advanced organization performance.

You will ensure that every person reaches true mastery* in the shortest amount of time. Mastery* produces people who have the adaptive reasoning skills required to effectively apply knowledge to new situations.

*Mastery is required where appropriate information application is critical to achieve a desired outcome; For example, Education: K-12, Higher, Corporate, Government; Medical; Medical Continuing Medical Education – Performance Improvement; Safety; Sales Performance; Aviation; Military; Individual Certifications and Individual Continuing Education.

McDonald Sales and Marketing, LLC, Specialties:

Appropriate Application;   Behavior Change;  eLearning;   Employee Selection and Employee Development;   Human Capital Management;  Individual Performance Improvement;  Learning Performance Improvement;   Learning Retention and Reinforcement Improvement;   Organization Performance Improvement;   Workforce Development and Productivity;  Verbal Skills Simulation and Reinforcement;   Sales Performance Improvement;   Web-Based (SaaS/Cloud) Solutions

To Discuss how these Solutions will add value for you, your organization and/or your clients, Affinity/Resale Opportunities, and/or Collaborative Efforts, Please Contact:

Tom McDonald, tsm@centurytel.net; 608-788-5144; Skype: tsmw5752