Learning: Virtual Observers in eLearning Leads to Returned Business Intelligence

Learning: By Peter Sorenson, President of QUIZZICLE 

New artificial intelligence embedded in eLearning solutions observes the actions of e-learners in real time. It records specific actions of the elearner—similar to how an instructor monitors a classroom. This virtual observer is able to identify when a learner is having difficulty and engages them in a type of “conversation” to assist and guide learning 

In 1954 Donald L. Kirkpatrick published his doctoral dissertation that included the early concepts that his Four Levels of Evaluation are based. Since that time, numerous theories have emerged attempting to justify the implementation and return-on-investment of eLearning in the corporate environment. But we needed a better way to observe the behaviors and knowledge transfer of those participating in eLearning solutions.  

Learning: Evaluating the traditional classroom 

We know when a learner attends traditional classroom-based training they are observed, and in a sense, evaluated by the instructor. An instructor learns about the effectiveness of her teaching method or style, and about how the class and individuals react, by observing behaviors in the classroom. The instructor notices and mentally records intent or effort, comprehension, and areas of struggle long before test scores indicate challenge or success. Good instructors use these observations to modify and refine their approach to engaging the learner with subject matter. 

Often in the classroom environment, learners, lesson plans, and entire curricula are evaluated based on observed learning behaviors and results. Evaluation is subject to both short and long considerations. 

In short-term evaluation, the effectiveness of the classroom presentation is judged in real time by appraising the level of interest displayed by the audience. Adjustments may be made by the teacher based on interplay between themselves and the learners. In many ways, moment-to-moment interactions guide the delivery of content to the learner body. 

A long-term evaluation requires a comprehensive review that considers many weeks of classroom presentations, quiz scores, assignments, and an overall assessment of learner comprehension. When performed, in-depth analysis enables instructors to revise teaching approaches or reference materials to fully engage participants and successfully address displayed interests or shortcomings of the group. It is a widely held theory that success is measured by how well learners engage and respond to a topic or concept as participation is an accepted indicator of learning. 

Evaluating web-based learning solutions 

Web-based training, as it is currently practiced in the corporate environment, is a uniquely isolating experience. Unlike the classroom, the opportunity to be recognized as an individual or to summon help when needed is unavailable. 

It has not been a goal of the eLearning application developers to include the support mechanisms inherent in traditional classroom training. In asynchronous training environments, a learner who experiences difficulty with a concept had no opportunity to request assistance. Likewise, course feedback often has been limited to “check-marks” noting completion, a test score revealing a simple pass or fail, or a post-course smile-sheet survey that delivers a subjective and biased review. 

Some learning management systems (LMSs) try to simulate the social structure of traditional classrooms by including asynchronous vehicles such as opinion posting threads and calendars of scheduled events. Absent from these e-learning environments is the immediacy of real time classroom support. With no apparent instructor there has been little opportunity for the interactive dialogue from which we learn best. Without a way to observe the interaction of learner with course and instructor there has been limited data from which meaningful evaluation of the curriculum could be accomplished. 

Most agree that the eLearning field has unfortunately been diminished rather than enhanced by the use of rapid development tools that convert static PowerPoint presentations into mediocre experiences attempting to “teach” online. Instructional design, the backbone of most successful training development, has been usurped in the quest for speed as PowerPoint plug-ins have attempted to become the developer’s creation facilitator.  

While application developers have surely met a need for “speedy development”, in doing so, they have neglected important aspects of online training tools. We need to reclaim the technical ability and skill to create interest, interaction, and opportunity for feedback; in essence a qualified learning experience through the use of robust tools. 

For years AICC and SCORM have served not only as stalwarts of validated communication with an LMS, but also as standard bearers in identifying the types of data-base records necessary for generating reports able to withstand the scrutiny of audits. These data records were never intended to indicate the quality of the user experience or to help in evaluating the competency of a course. 

Enter the virtual observer 

The eLearning industry must lead the way in moving us beyond the idea that web-based training is simply a means to “maintain compliance” and toward the realization that we can facilitate meaningful learning experiences in a web environment. But how? By embedding artificial intelligence technology into the eLearning solution to observe the actions of elearners engaged in web-based training in real time. It records specific actions of the elearner—similar to how an instructor monitors a classroom.  

This virtual observer is able to identify when a learner is having difficulty and engage them in a type of “conversation” to assist and guide learning. This artificial intelligence also can independently decide to suggest modifications to the learner’s course interactions to actively encourage behaviors that increase comprehension, rather than passively allow the simple completion of tasks. By referencing learner’s previously recorded training behaviors from their individualized learning profiles, our technology has the potential to assist in further personalizing the training environment. We are able to provide real time guidance to help the learner achieve greater success in learning. 

 

Next, a robust reporting tool provides training managers a means to access the collective experiences of all learners to identify successes and failures of the course itself. Likert ratings and dynamic graphs provide a simple yet effective way to identify which pages, concepts or approaches proved most successful in supporting comprehension and which proved more challenging.  

With hard behavioral data as feedback, the instructional designer or developer is able to make modifications to effectively improve their courseware, and monitor the impact of their modifications by reviewing the experiences of future learners. Successful design approaches are revealed from learner data and the goal of teaching, which is to facilitate learning, more easily accomplished. 

Beyond providing robust learner support and rich evaluation capabilities, it is important to note that, for the first time, tangible data in the form of business intelligence is being returned to the company or client funding the training. This data can be used by business analysts to answer questions that were not possible to address previously due to an absence of available information. It is now possible for training departments to become familiar with statistical terms such as clustering and outliers, and use this knowledge and understanding to contribute to the development of more successful training courses.

 

Bottom line: RBI 

Before such artificial intelligence, the eLearning community relied on “theories” to support the realization of ROI from web-based training development budgets. We know that to capture, record, and analyze the behavioral data of our learners provides greater meaning and substance to the claim of an ROI benefit as returned business intelligence (RBI).  

We have, in essence, created a positive, self perpetuating cycle that will improve the quality of web-based training. To capture mineable data through learner experiences, we as developers may now begin to include many diverse opportunities for learners to observe actions that reveal their strengths, shortcomings, and learning styles. Instructional designers may now fuel the course with more robust instructional components, and learners in-turn will fuel the database with their responses captured as data, allowing managers to review how learners interact with course material page by page.  

This model of watch and coach is widely held as an excellent approach for learning. For instance, it is the reason coaches videotape their athletes. It is the reason the instructor stands in front of the classroom. Observing real-time behaviors and patterns enables an instructor to appropriately intercede, redirect, and provide a variety of specialized interventions to support learner success. This method far surpasses the subjective “end of class survey” in delivering unbiased, useful feedback from which strengths and weakness of course and learner may be addressed.  

Demonstrating fiscal responsibility, training departments can substantiate their need for the larger budgets required to develop robust interactive training through the use of predictive and advanced analytics. By revealing group learning patterns and individual behavioral dynamics, businesses can access data that was never before available to help them make informed business decisions. The possibility exists for HR to extrapolate learning and behavioral tendencies to effectively pair employees with job roles for better placement or job satisfaction. 

Using these technologies, eLearning is poised to influence businesses and employees in ways never before considered by simply employing the same technique that teachers and coaches have used with great success. Watch the learner.

http://www.astd.org/LC/0911_sorenson.htm 

Peter Sorenson is the president of QUIZZICLE. Involved in designing and developing web-based training since 1995, his goal is to programmatically replicate the classroom experience within the online environment. Contact him at peters@quizzicle.com.

==================================================================

To Discuss how these Solutions will add value for you, your organization and/or your clients, Affinity/Resale Opportunities, and/or Collaborative Efforts, Please Contact:

Tom McDonald, tsm@centurytel.net; 608-788-5144; Skype: tsmw5752