How to Create First-Rate Training Reinforcement Content

Download

Content is the fuel that powers every highly effective learning reinforcement plan (LRP). Generating reinforcement content is not difficult, but it is different from course development. Here are the insights we’ve gained through our creation of reinforcement content for companies in retail, manufacturing, IT and hospitality. It all starts with extracting key concepts from the very course that needs to be reinforced.

Your High Performance Training Reinforcement Plan Needs High-Octane Content

Any learning and development (L&D) professional knows how challenging it is to get thousands of employees up to speed on a new product or skillset – quickly! The design and delivery of an initial training program (or course) is one part of the challenge. The next is creating incentives that motivate the employees to complete the course. And finally, you need to ensure employees retain this new knowledge by reinforcing the competencies they acquired in the course. Think of these as the training challenge, the incentive challenge and the retention challenge.

As L&D professionals, we often overlook the retention challenge at our own peril. When we do, the “forgetting curve” starts to erode our hard-earned victories in overcoming the training and incentive challenges. And this issue is further compounded by illusions of knowing. (To learn more about these concepts and how they lead to workplace knowledge gaps, take a look at “How to Overcome the Illusion of Knowing” and “How the Mobile Revolution can Overcome the Forgetting Curve.”)

Numerous corporate training success stories (see “GE Achieves a 30% Boost in Learning Retention”) show that an effective method to tackle the retention challenge is for newly trained employees to enroll in a Learning Reinforcement Plan (LRP) immediately after they’ve completed their initial training. We say immediately, because the forgetting curve is merciless. It starts to erode employees’ acquired learning even before the training event is over.

After you determine which training programs can benefit from reinforcement (see “Where to Focus Your Training Reinforcement Efforts”) and decide on a basic structure, your next consideration is what goes into your LRP – in other words, the LRP’s content.

Content is what makes or breaks an LRP. An LRP that has a perfect structure and schedule but mediocre content is an LRP that stalls.

The good news is that developing effective content for most LRPs doesn’t need to be complex or time consuming.

It all Starts with the Initial Course

Employees don’t use LRPs to relearn what they’ve forgotten – they use LRPs to stop forgetting what they’ve already learned so they can transfer it effectively to their jobs. Training reinforcement is not relearning. LRPs are defensive in nature. They protect the acquired knowledge from the menace of forgetting. They are not intended to instill new learning.

If LRPs are designed to reinforce what has already been learned, the initial course is where that learning can be found. So the secret ingredients for great reinforcement content resides in the initial course that your LRP aims to reinforce.

The idea isn’t to reinforce everything that’s been taught and learned in that initial course. To be effective, your LRP needs to reinforce the core structure of the original course that is most fundamental for long-term retention and job performance.

Because creating an LRP primarily involves reusing or repurposing the content that’s already been created, assembling a typical LRP costs a fraction of the initial training development investment.

Retrieve to Retain

Effective training reinforcement relies on retrieval practices. These practices make newly acquired learning long-lasting and help bridge the learning-performance gap. Developing great reinforcement content, to a great extent, comes down to developing great retrieval practices.

Retrieval practices are short (typically 5 minutes) and periodic (every day or every other day) activities during which employees recall previously learned concepts from memory. Periodic retrieval of the knowledge that has recently been learned helps employees move that knowledge to long-term memory.

While retrieval practices can take many forms, the most common form in today’s technology-delivered LRPs is the multiple-choice question (MCQ). Often equipped with a variety of advanced features such as an intelligent behaviour that adjusts to each learner’s competency gaps, MCQs are effective, easy to create and flexible. They are the silent champions of successful corporate LRPs.

A word of caution is warranted, however. Throughout this discussion, we are referring to the MCQ as a practice tool. Don’t confuse this with the other, and more traditional, use of the MCQ as an assessment tool. In the world of training reinforcement, trainees use MCQs to make their learning durable, not because someone in the organization is trying to assess what they do or do not know.

A 5 Step Methodology

Here are 5 simple steps you can follow to develop the content of your learning reinforcement plan.

Step 1: Identify the Core Competencies

You don’t need to reinforce every concept from the initial course. Instead, focus on the training’s core ideas and concepts that form the most critical knowledge your employees need to transfer to their everyday jobs. We will refer to these as core competencies. Your LRP needs to reinforce only these competencies.

Most LRPs have between 5 and 15 competencies. To the extent that you arrive at a much larger number of competencies, you may be defining your competencies at too granular a level.

One way to do this is to focus on those concepts that employees need to be able to readily recall and apply on the job. Think of Captain Sully, who, after losing both engines, safely landed U.S. Airways flight 1549 into the Hudson River. He had honed the core skills needed to land the plane over years of continuous training and simulation. On the day of the incident, he didn’t have the time to look up the procedures for a water landing. Instead, he was able to readily recall and apply knowledge he had acquired many years ago and retained since then.

What concepts will your employees need to know, and more importantly, instantly recall, while performing their jobs? What matters the most in their day-to-day tasks? What procedures do you want to become second-nature to them?

Step 2: Develop a Solid Question Bank

Expect to spend more than 80% of your content development time on design of the MCQs in your LRP’s question bank. Each 5-minute daily practice session pulls (serially, randomly, or preferably, dynamically) a few questions from this bank and serves them to the learner.

*See the sidebar for tips on creating MCQs that help retention.

Step 3: Define Bite-Sized Learning Resources

The purpose of an LRP is to reinforce the learning that employees have recently acquired in a structured course or training program. Since the LRP is not intended to teach new material to employees, its learning resources are limited to what the employees have already been exposed to and covers only the competencies you’ve selected. These all come from the original course.

Learning resources can take a variety of forms: micro learning modules, PDF summaries, podcasts, short videos, etc. In all cases, learning resources need to be available to learners in bite-sized form (consumption time of no more than 5 minutes) and, ideally, packaged in a manner that employees can easily access from their mobile devices, even when they don’t have a network connection.

Most LRPs avoid creating new learning resources and instead rely on a referenced, repackaged or repurposed version of the material from the original course. To some extent, your budget determines the extent to which you can repurpose the course content for reinforcement use. It’s important to avoid the temptation to make your LRP a full-fledged course. Don’t forget, the retrieval practices are the heart of your LRP. Learning resources play only a secondary role.

Step 4: Map the Questions and Learning Resources to Competencies

Map each MCQ in the question bank you created in step 2 and each learning resource you identified in step 3 to one or more of the competencies that were defined in step 1. You now have a cohesive reinforcement plan that can equip your employees with the enduring knowledge that their jobs require.

This mapping allows you to determine whether each of the identified competencies in your LRP has adequate coverage in terms of retrieval practices and learning resources. You may want to run your LRP through our suggested Pre-launch LRP Checklist before you put it into action.

Step 5: Deploy, Measure, Fine Tune, Repeat

Pilot your content with a small group of trainees upon completion of your training program. It is you who is now in learning mode. Use the analytics or business intelligence (BI) tools at your disposal to measure the effectiveness of the program for this first cohort of employees. Check out “Reinforcement Metrics: How to Use Analytics to Optimize the Effectiveness of Your Learning Reinforcement Plan” for a comprehensive overview of reinforcement metrics.

As you gain insights into possible steps to improve your content, you need to decide when to implement these steps. Some corrective actions can be made midstream while the first cohort of trainees is progressing through the LRP. Others may need to be implemented for subsequent cohorts (intra vs. inter LRP adjustments). Perhaps you need to adjust the metrics you’re using to monitor the progress of your content.

You have now officially entered the deploy-measure-refine-repeat cycle. Think of this as a continuous improvement endeavor that doesn’t really end. The net result is a top performing LRP that is fuelled by first rate content.


Tips for Developing Multiple Choice Questions that Help Retention

Multiple Choice Questions (MCQs) are a common form of retrieval practices in most corporate LRPs. Here are a few considerations to help you define your MCQs.

Question Bank Size

Your question bank needs to have several questions for each competency covered in your LRP. Although your budget will partly determine how many questions you include in your question bank, you need enough questions to ensure adequate coverage and variety for each concept. A small question bank is a common symptom of low-performing LRPs. Too few questions on each competency may lead to learners mistaking familiarity and fluency with long-term retention, and it may even cause them to disengage from your LRP (see “How to Overcome the Illusion of Knowing with a Mobile Reinforcement Plan” for more).

A cost effective way to expand the size of your question bank is to define a few “foundational” questions for each competency and multiple “derivative” questions from each of these. Developing the derivative questions shouldn’t take much effort.

Knowledge and Scenario-Based Questions

Scenario-based questions, such as “What would you do in this situation?” are a particularly effective way to reinforce what employees have learned in classroom or web-based courses. Usually there isn’t enough time in a structured course to engage trainees in scenario-based practices. In contrast, a reinforcement plan’s longer timeline and inherent flexibility offer an excellent opportunity to present scenario-based questions.

Scenario-based questions bridge the gap between “learning” and “doing”. The more employees envision themselves in a particular situation, the better they’ll deal when it arises in real life. For example, if you’re developing an LRP for your sales training program that includes the competency “finding the need behind the need”, ask trainees to find the need behind the need of a customer who says “I need an expense tracking app”.

Difficulty Level

Retrieval practices need to be effortful to be effective. Your questions need to strike the right balance on the difficulty scale. Trivial questions may help learners rank higher on leaderboards (in the case of a gamified LRP), but they won’t help learners achieve long-term retention of key concepts. Now, you also need to be careful that you don’t make the questions so difficult that you unintentionally harm learners’ engagement.

What and how many answer choices and hints you develop for each MCQ is one way to adjust the desired difficulty.

To determine whether or not you’ve struck the right balance, use analytic tools to spot those outliers whose difficulty level needs to be adjusted up or down. This topic is explored further in “How to Use Analytics to Optimize the Effectiveness of Your Learning Reinforcement Plan”.

Feedback

Feedback, which often gets overlooked, is one of your reinforcement plan’s best assets. Feedback goes beyond whether the learner has answered an MCQ correctly or not, and can include meaningful explanations that truly help reinforce a key concept. You may also want to include links in your feedback that point to learning resources the learner should review to reinforce the concepts targeted by the question.

Reinforcement systems can be configured to provide feedback immediately after the learner has attempted a MCQ (immediate feedback) or push that feedback to the end of the daily reinforcement session (delayed feedback). And feedback can be provided at the question level or answer choice level.


Your Pre-launch Content Checklist

  • The core competencies for my LRP are clearly defined.
  • I have at least a few MCQs for each LRP.
  • I have defined multiple MCQs for the more important competencies.
  • The question and answer choices of each MCQ strike the right difficulty level.
  • I have developed feedback statements for each question and answer choice.
  • I have mapped each learning resource to one or more competencies.
  • I have defined the metrics that I plan to use to measure the effectiveness of my LRP.

Other LRP Content Types

MCQs and learning resources, in that order, form the most essential elements of your LRP content. While the contents of many highly effective LRPs are limited to these elements, your LRP can also include one or more of the following content types:

Flashcards

Flashcards are a highly effective form of retrieval practice. Technology-based LRPs support flashcards that include graphics, audio, video and interactive content. Flashcards can be predefined by LRP designers or created by the learners themselves to suit their particular needs. They can also leverage a number of training reinforcement principles, such as feedback, interleaving and variation.

A good way to start with flashcards is to define an initial deck of cards for one of the competencies in your LRP and allow learners to use that as the basis for adding their own cards or creating their own decks for other competencies.

Smart Tips

Advanced training reinforcement platforms allow you to serve daily tips that reinforce learners’ understanding of some or all of the LRP’s competencies. These tips can be “smart” in the sense that the daily selection is determined according to the learner’s progress, targeting competencies that he or she is struggling with.

Gamification: Awards, Competition and Leaderboards

Your LRP can include elements that address the issue of learner engagement. Without engaged learners, your LRP is not likely to succeed, irrespective of how many excellent retrieval practices, learning resources or smart tips it may include.

Adding gamification elements is one way to boost engagement. Gamification can take the simple form of including awards and badges (for effort or outcome) or a more sophisticated form that entails individual and team competition and leaderboards. Choose the format that best matches your company’s culture. Always start small and test your way up towards more engaging and effective reinforcement content.