Instructional Triage: Getting the Right Help to the Right Learner...Fast!

One way to think about an education system it to think of it as a race against time for mastery.  There are only so many instructional hours and learners have so many different needs, some of the easy to address, some harder to address, and some that need a comprehensive intervention.  This way of framing an educational system made me think of a medical triage process during a large scale disaster.  The analogy goes like this:

You have a lot of people who need a lot of different kinds of help.  You can’t ignore any of them, because they all need your intervention, but you must to do two crucial things:

1.       You must reduce the average time to help for everyone in need to be as low as possible so no one dies simply because you haven’t gotten to them yet (due to the crowded, chaotic conditions).
2.       You must apply your most expert, most scarce, and therefore most expensive resources (expensive in time, salary, and sophistication) as precisely as you can to those who truly need them the most.

While not crucial in an emergency (no one is going to die immediately over this one), you have another goal:

3.       You should match the expertise of the other resources you have as precisely as you can to the degree of need.  You don’t want an RN treating someone who only needs a CNA or vice versa, because it is wasteful on the one hand, and on the other could turn a less severe need into a more severe one because you didn’t apply the right expertise/remedy soon enough.

To accomplish the above, you need to have a system for most of the people that provides an ever widening scope and degree of intervention (start with a first aid kit for self-treatment and work your way up) while quickly identifying those with severe needs that the other solutions aren’t going to be able to address and pulling them out of the queue and moving them directly to the comprehensive help they need.

Just about every organization that is responsible for helping others learn is in a similar race against time to get the right help to the right people in what can be an environment of scarce teaching/tutoring resources (making it "crowded") and with limited, sometimes conflicting information about the exact and diverse needs of each learner (making it "chaotic").  

So applying the principles of triage to a learning organization:

1.       You can reduce the average time it takes to get help in these ways:

a.       The more you know about the learner coming in, the quicker you are going to be able to identify needs and issues.  Every scrap of demographic information, every bit of data from their previous learning experiences, and any surveys or diagnostic tests they complete at sign up is all going to help you figure out their needs more quickly. 

IMPLICATION #1: It should be a priority to collect as much background information as possible on each learner and form predictive algorithms around it that are constantly refined and improved with the goal of anticipating their needs from day one.

b.      The smaller the chunk of meaningful learning you can assess and the quicker you can do so, the sooner you are going to be able to identify issues and needs.

  IMPLICATION #2: We should prioritize the creation and deployment of vastly more formative, granular assessments in our learning materials that are focused on assessing key concepts, the mastery of which are demonstrated to lead to essential learning outcomes.

c.       If you can identify major risk factors early on, you can save the inefficiencies of applying remedies that are not adequate and you can increase the amount of time the learner has exposure to the right remedies.

 IMPLICATION #3: We should become experts at identifying major risk factors very quickly and escalating the interventions immediately.

2.       You can apply expert resources precisely in these ways:

a.       You first must make sure that your instructors/mentors are undistracted so that their very expensive, expert time is not spent on paperwork or other administrative matters.

 IMPLICATION #4: It should be a priority to optimize the technology and administrative systems such that instructor time spent doing anything other than teaching/intervening with learners is as close to zero as possible and those tasks that are necessary but don’t lead directly to improved learning outcomes are delegated to others/automated.

b.      You must have experts that truly understand the proper, most effective interventions and how to apply them. 

 IMPLICATION #5: It should be a priority to hire staff with expertise in maximizing learning outcomes in a variety of challenging circumstances, to provide professional development to those who are expected to apply the expertise, and to run systematic pilots to figure out what to do when no one knows the answer and then operationalize and scale these interventions.

c.       You need to provide your instructors with the necessary diagnostic data so they can be efficient and accurate in their intervention. See 3.b. below.

3.       You can match expertise to degree of need in these ways:

a.       First, you must have an ecosystem that has a variety of interventions to apply.  If you only have a hammer, you are only going to pound things.  We need a diverse toolbox from which to choose.

 IMPLICATION #6: We need to develop diverse, calibrated, escalating, mutually aware, agile instructional interventions that are systematically applied if and when the previous one proves to be inadequate for timely mastery.

b.      You must have real-time, reliable data that lets you know which intervention to apply to whom. 

IMPLICATION #7: We need a highly sophisticated learning analytics system that measures learner progress velocity and mastery in real-time and can either automatically apply interventions reliably and intelligently or prompt a instructor/mentor to select an intervention from a vetted and targeted set of options to apply.

As an incredibly useful byproduct of the above, this system will generate an auditable trail of evidence of learning and mastery on a per learner basis that can be used to show all kinds of internal and external stakeholders exactly what the learner knows and how they learned it and we can use the same audit trail to teach ourselves how to teach the learners better.

Real-Time Business Case Method: Study What a Real Company is Doing. Right now. This Minute.

Since 1920, the basic case study method used first at Harvard Business School (HBS) and then by practically every business school on earth has changed little. Sure, they are putting them on tablets now, but otherwise the simple formula is 1. research something that happened, 2. write it up, 3. leave the key issues open for the students to analyze and interpret (even though the outcome is known to the case designer), and 4. hold a highly accountable class discussion about it. Often at the end, the outcome is revealed for comparison to the students prescriptions or predictions.  Nice. Tidy.  Print, repeat.

And print they did.  HBS alone has 17,000 case studies for sale!

But what if you took the whole concept up a notch?  What if the outcome was unknown because it simply hadn't happened yet?  Because the company was making the crucial decisions right now?  Today and tomorrow and throughout the course and, if the company survived, long after the students left the classroom.

And what if you had a front seat to the action? A reporter inside the company documenting what was happening week by week, interviewing executives and employees, and sending out updates once a week, including company documents, strategy, and financial forecasts?

And what if you could actually speak to the executives via video conference and ask them questions once a week?  And what if you were allowed to critique their business strategy using the same information they have available to them and provide recommendations for their consideration?

This is precisely the audacious experiment conceptualized and implemented by James Theroux at the Isenberg School of Management.  The results were phenomenal.  In the first implementation of this approach, 45% of  the students said they felt it was the most memorable or valuable course they had taken.

And was it good for the company?  Why would they air their dirty laundry in front of and answer questions from a bunch of students when they have urgent work to do and critical decisions to make?  Well, consider that the students identification of core weaknesses in the business model turned out to be accurate and their recommendations for action prophetic.

According to Theroux:

“Over time, and including [a] merger, the result has been to a large degree that [the students'] thoughts on positioning the company have come to fruition.”

Repeating this kind of success is difficult.  In a later iteration, a mere 25% of the students found it to be the most memorable or valuable course they had taken (I hate it when only a quarter of my students think my class was the best one ever!). The company in this case was slower moving, the technology field it was in was harder to understand, and the access to the leadership was, unexpectedly, more limited.

But the best things to do are way too often also the hardest things to do.  It is worth trying again and again to get a model like this right; to refine the selection criteria; to get the costs of engaging in the approach manageable (either by lowering execution costs or by scaling participation fees); to demonstrate benefits so compelling to both the company and the learning institution that the approach becomes accepted and routine and takes it place along side that other, once radical, approach that has been working since 1920.

Hats off to Dr. Theroux!

For more information on Real-Time Case Method, see:
Real-Time Case Method: Analysis of a Second Implementation. Theroux, James M.
Journal of Education for Business Jul/Aug 2009, Vol. 84 Issue 6, p367-373 7p; 2 Charts, 12 Graphs, 2009 

The Power of Learning Analytics, Targeted Intervention, and Mastery Learning at Scale

This is the first time I have ever reblogged, but the posting by Bror Saxberg, CLO of Kaplan, regarding applying learning analytics at scale at the University of Wisconsin at Milwaukee is worth repeating.  It is a powerful combination of large scale analytics, careful experimental design, targeted intervention,  semi-automated student motivation, and mastery learning. Make sure you jump to the original via the link below to see some very compelling charts representing their success.

By the way, I ran in to Bror at the Educause Learning Initiative conference, and he is more than ready to lead the revolution for the application of learning sciences to higher education, and he has enough energy for a dozen mere mortals to pull it off!  I am interested in connecting with others who share this passion as well. 

Bror writes:

At the recent Educause Learning Initiative meeting in Austin Texas, I came across some very interesting recent randomized control trial results from the University of Wisconsin at Milwaukee.  They’re testing a combination of mastery learning plus “amplified assistance” (data-driven suggestions for faculty about who to intervene with and how) in an introductory Psychology course, and, with thousands of students (!!) having run through controlled trials, they’re showing significant improvements in pass rates and long term retention.

via brorsblog.typepad.com

Adventure Learning: Real Time, Real People, Real Issues

Imagine teaching biology to a group of students in a typical classroom and beginning class this way:

And now we will chat with a scientist on an expedition in Greenland answering question we have about changes in the ecology due to warming climate trends.

Or imagine learning about sustainability while following a team of scientists and educators through Africa, speaking with local villagers about sustainable agricultural practices or the impact of globalization. These are both possible thanks to some incredibly forward thinking and fearless innovators at the University of Minnesota. They call this hybrid distance education approach, Adventure Learning.

To quote from their wikipedia article:

Adventure Learning (AL) ^[1] is a hybrid distance education approach...[that provides] students with opportunities to explore real-world issues through authentic learning experiences within collaborative learning environments, and is anchored in experiential and inquiry-based learning.^[2] The AL approach includes educational activities that work in conjunction with the authentic experiences of researchers in the field. For example, within an AL program, the curriculum, the travel experiences and observations of the researchers, and the online collaboration and interaction opportunities for participating learners are delivered synchronously so that learners are able to make connections between what is happening in the real world and their studies, and then reflect on those events and present potential solutions to issues that are raised.^[3]

The real world is happening 24/7 in every country on every continent on the planet. Why is it that when we enter a classroom setting, or even, unfortunately, when we enter most online courses, we shutter the windows as it were to the rest of the world. Why don't we invite that world in? Why don't we make the human connections that are possible now between what we are learning and the people that live those lessons every day?

Way back in the mid-90s, I ran an internet science camp for middle school kids. They were marginally engaged. Then one day we sent an invitation to an online group of scientists asking if they would chat with our class. To our great surprise, we had about 10 respond with an enthusiastic YES! I can still remember the day that our students were chatting live with scientists who were designing the Mars rovers or researching at Microsoft or DuPont. Our students were completely engaged, and everything we were trying to teach them about a future career in science was demonstrated in spades by these willing professionals reaching out to them with answers and advice.

How far can we take this approach? Imagine a language learning class following peers of the same age traveling through a country speaking the target language and engaging the target culture or just connecting with a class of local students. Imagine a science class connected to a team designing a solar car for the World Solar Challenge. Imagine an ancient history class following an excavation at Petra.

The possibilities are endless. Yes, it takes some extra effort. But, wow, isn't it worth it?

The Chocolate and Peanut Butter of Learning Analytics

On October 23rd, 2009, Adobe Systems, Inc. competed its acquisition of Omniture, Inc. An Adobe press release describes the benefits of this merger as follows:

The combination of the two companies will increase the value Adobe delivers to customers. For designers, developers, and online marketers, an integrated workflow — with optimization capabilities embedded in the creation tools — will streamline the creation and delivery of relevant content and applications. This optimization will enable advertisers and advertising agencies, publishers, and e-tailers to realize greater ROI from their digital media investments and improve their end users' experiences.

Beneficiaries: advertisers, advertising agencies, publishers, etailers
Benefit: greater ROI, better end user experience

What this press release does not say is that on that fateful day the chocolate of Omniture's web analytics tools dropped smack dab into the middle of the peanut butter of Adobe' s eLearning suite. Or it could have/should have/will if someone stops trying to flip up the most enticing Flash banner ad possible for a minute and thinks about the eLearning world. If the red and black eLearning folks will wander over to their new 1.8 billion dollar, lime green roommates and make a modest proposal, we could have a match as classic and enticing as the Reece's peanut butter cup.

Does Adobe have any idea that they are sitting on the biggest revolution in eLearning since the browser? Do they realize that they now have at their finger tips all of the tools necessary to dominate the eLearning space with the hottest, most completely integrated, most elegantly implemented learning analytics suite on the market? Who else has the power to build learning analytics straight into the most popular tools for elearning design and development? Who else has the statistical and number crunching guns to process and display massive amounts of learner data in slick, easy to use dashboards? Could there be a more obvious fit? Does anyone there realize that they could be the engine that powers a massive emerging industry?

Probably not. Why would the people focused on ROI for advertisers start scribbling on the back of napkins with people who are focused on ROI for eLearning ? Nothing against Adobe; big corporations just don't innovate this way very often. If they did, the press release might read as follows:

The combination of the two companies will increase the value Adobe delivers to learners everywhere. For instructional designers, eLearning developers, and online colleges and universities, an integrated workflow — with learning optimization and tracking capabilities embedded in the creation tools — will streamline the creation and delivery of customized learning content and experiences. This optimization will enable teachers, trainers, instructional designers, and training organizations as well as online educators to realize greater ROI from their digital training and teaching investments and improve their end users' experiences and, ultimately, the overall appeal, effectiveness and efficiency of their learning.

It would be a crying shame if this didn't happen. Imagine a world without Reece's peanut butter cups. It would be that bad.

Johnny Depp Kills Disney Pirates

Having just returned from Disneyland, I have been thinking about the updated Pirates of the Caribbean ride that now features Johnny Depp as Captain Jack Sparrow. While Disney surely added Capt. Jack to the ride to update it and increase audience engagement, in discussing it with my wife, it became clear to us that the addition actually reduces the narrative power and, thus, the overall engagement and enjoyment of the ride.

Timelessness

What makes good narrative great is when it is imbued with a sense of timelessness. While stories like Huckleberry Finn or To Kill a Mockingbird have a definite time and place, they don’t feel bound to that particular time. The characters feel fresh and real every time you read them, and the settings feel like they could exist right now in some parallel universe coexistent to our own. Before the addition of Jack Sparrow, the ride had this feeling to it. The riders filled in the details of the narrative, imagining a back story and a conclusion, projecting the motivation of each character and their ultimate fate, placing the events in an imaginary, but plausible, history. With the addition of Sparrow, all that is taken away. The narrative is forced (rather ungracefully in its repetition) upon you. You are now watching a version of a blockbuster movie, and you know the back story and the ending because it has been thrust upon you. The narrative has been appropriated, and you must comply.

Rider as a Character

Even worse is the loss of the inclusion of the riders as characters in the story. In the original ride, you become a part of the story. You are warned at the beginning that you are entering into danger, you then view the “cursed treasure,” and are then told the following:

“No fear hath ye of evil curses, says you. Ah... Properly warned, ye be, says I. Who knows when that evil curse will strike the greedy beholders of this bewitched treasure.

Perhaps you/ye knows too much. You've seen the cursed treasure. You know where it be hidden! Now pass through at your own risk. These be the last friendly words you'll here!”

You are now cursed and must face your fate. That curse brings you right into the middle of a fierce battle with cannon balls whizzing overhead, then through a burning, groaning, crumbing building, and finally between the muskets of drunken, dueling pirates. Happily, you survive all of these and leave with your secret knowledge of the treasure and your lives. All this is taken away from you in the new version. You simply watch as other characters play out other stories. The boat is little more than a floating theater seat. Pass the popcorn.

Implications for Instruction

Seeing or hearing a narrative is a powerful means of engagement. Even more powerful is direct participation in the narrative by taking on a role and/or being asked directly or indirectly to project the conclusion. People like Stephen Covey teach using great narratives. But they keep those narratives for themselves because they provide all the pieces and trot them out in each training masterfully and verbatim like an actor on a stage. While this is so much better than PowerPoint bullets, it is not as powerful as sharing the narrative with the audience, giving them a real stake in its outcome, and even trusting them to write the ending.

Unforgetable Learning: What Master Storytellers Can Teach Us About Designing Transformative Learning Experiences

This post is the story of the dissertation I didn't write, the book I would like to edit to make up for it, and how you can help.

Have you ever been channel surfing and came across a documentary about a topic that, until this pivotal moment, was of mild interest and found yourself, within 10 minutes, fully engrossed in every aspect and detail of say, the life cycle of a shrew, the mating rituals of the common slug or the travails of a nomadic Mongolian family whose camel has rejected her calf? Do you find yourself telling your co-workers the next morning with great enthusiasm about the way computer chips are manufactured or how FedEx sorts packages or how sea turtles migrate thousands of miles to the exact same beach on which they were born to lay their eggs having never visited it again in the intervening decades?

My dissertation was going to be a deep study of the structure of the most engaging, memorable documentaries designed for a general audience. I wanted to extract principles of how to engage and enthrall learners who only have a passing interest in a subject. It seemed to me that our field could learn a lot from those outside the field who had mastered this craft. Alas, I couldn't muster enough interest among my faculty and had to move on to another topic.

Master storytellers design their narrative to completely invest their audiences in their story; and audiences willingly give their rapt attention and full emotional involvement in return. I want to know how that is accomplished by the best storytellers in (at least) the following genres:

• Documentaries
• Feature Films
• Television Series
  
• Plays
• Musicals
• Dance Performances
• Poetry
• Short Stories
• Novels
• Video Games
• Role Playing Games
• Theme Park Rides

I believe much could be learned from a minute by minute analysis of the structure of some exemplars, by interviewing their creators, by interviewing their loyal audiences (or eavesdropping on their public online discussions), and by carefully studying those who experience these for the first time. I also believe that the narrative traditions, folk wisdom, and design tools of the individual genres could be mined for insight. For example, I did a minute-by-minute analysis of the NOVA episode on the Wright Brothers and found very clear and compelling structures designed to draw in and hold audiences with different levels of familiarly and interest in the topic.

I would love to serve as editor or co-editor on such a book, with chapters written by you or people you know who would be ideal candidates. I would be interested to hear from those who would like to participate or who know people who would like to submit a chapter or could recommend a publisher that would find such a book particularly interesting. Actually, I would just plain like to know what you think of the idea.

What do you think?