Instructional Triage: Getting the Right Help to the Right Learner...Fast!

One way to think about an education system it to think of it as a race against time for mastery.  There are only so many instructional hours and learners have so many different needs, some of the easy to address, some harder to address, and some that need a comprehensive intervention.  This way of framing an educational system made me think of a medical triage process during a large scale disaster.  The analogy goes like this:

You have a lot of people who need a lot of different kinds of help.  You can’t ignore any of them, because they all need your intervention, but you must to do two crucial things:

1.       You must reduce the average time to help for everyone in need to be as low as possible so no one dies simply because you haven’t gotten to them yet (due to the crowded, chaotic conditions).
2.       You must apply your most expert, most scarce, and therefore most expensive resources (expensive in time, salary, and sophistication) as precisely as you can to those who truly need them the most.

While not crucial in an emergency (no one is going to die immediately over this one), you have another goal:

3.       You should match the expertise of the other resources you have as precisely as you can to the degree of need.  You don’t want an RN treating someone who only needs a CNA or vice versa, because it is wasteful on the one hand, and on the other could turn a less severe need into a more severe one because you didn’t apply the right expertise/remedy soon enough.

To accomplish the above, you need to have a system for most of the people that provides an ever widening scope and degree of intervention (start with a first aid kit for self-treatment and work your way up) while quickly identifying those with severe needs that the other solutions aren’t going to be able to address and pulling them out of the queue and moving them directly to the comprehensive help they need.

Just about every organization that is responsible for helping others learn is in a similar race against time to get the right help to the right people in what can be an environment of scarce teaching/tutoring resources (making it "crowded") and with limited, sometimes conflicting information about the exact and diverse needs of each learner (making it "chaotic").  

So applying the principles of triage to a learning organization:

1.       You can reduce the average time it takes to get help in these ways:

a.       The more you know about the learner coming in, the quicker you are going to be able to identify needs and issues.  Every scrap of demographic information, every bit of data from their previous learning experiences, and any surveys or diagnostic tests they complete at sign up is all going to help you figure out their needs more quickly. 

IMPLICATION #1: It should be a priority to collect as much background information as possible on each learner and form predictive algorithms around it that are constantly refined and improved with the goal of anticipating their needs from day one.

b.      The smaller the chunk of meaningful learning you can assess and the quicker you can do so, the sooner you are going to be able to identify issues and needs.

  IMPLICATION #2: We should prioritize the creation and deployment of vastly more formative, granular assessments in our learning materials that are focused on assessing key concepts, the mastery of which are demonstrated to lead to essential learning outcomes.

c.       If you can identify major risk factors early on, you can save the inefficiencies of applying remedies that are not adequate and you can increase the amount of time the learner has exposure to the right remedies.

 IMPLICATION #3: We should become experts at identifying major risk factors very quickly and escalating the interventions immediately.

2.       You can apply expert resources precisely in these ways:

a.       You first must make sure that your instructors/mentors are undistracted so that their very expensive, expert time is not spent on paperwork or other administrative matters.

 IMPLICATION #4: It should be a priority to optimize the technology and administrative systems such that instructor time spent doing anything other than teaching/intervening with learners is as close to zero as possible and those tasks that are necessary but don’t lead directly to improved learning outcomes are delegated to others/automated.

b.      You must have experts that truly understand the proper, most effective interventions and how to apply them. 

 IMPLICATION #5: It should be a priority to hire staff with expertise in maximizing learning outcomes in a variety of challenging circumstances, to provide professional development to those who are expected to apply the expertise, and to run systematic pilots to figure out what to do when no one knows the answer and then operationalize and scale these interventions.

c.       You need to provide your instructors with the necessary diagnostic data so they can be efficient and accurate in their intervention. See 3.b. below.

3.       You can match expertise to degree of need in these ways:

a.       First, you must have an ecosystem that has a variety of interventions to apply.  If you only have a hammer, you are only going to pound things.  We need a diverse toolbox from which to choose.

 IMPLICATION #6: We need to develop diverse, calibrated, escalating, mutually aware, agile instructional interventions that are systematically applied if and when the previous one proves to be inadequate for timely mastery.

b.      You must have real-time, reliable data that lets you know which intervention to apply to whom. 

IMPLICATION #7: We need a highly sophisticated learning analytics system that measures learner progress velocity and mastery in real-time and can either automatically apply interventions reliably and intelligently or prompt a instructor/mentor to select an intervention from a vetted and targeted set of options to apply.

As an incredibly useful byproduct of the above, this system will generate an auditable trail of evidence of learning and mastery on a per learner basis that can be used to show all kinds of internal and external stakeholders exactly what the learner knows and how they learned it and we can use the same audit trail to teach ourselves how to teach the learners better.

Real-Time Business Case Method: Study What a Real Company is Doing. Right now. This Minute.

Since 1920, the basic case study method used first at Harvard Business School (HBS) and then by practically every business school on earth has changed little. Sure, they are putting them on tablets now, but otherwise the simple formula is 1. research something that happened, 2. write it up, 3. leave the key issues open for the students to analyze and interpret (even though the outcome is known to the case designer), and 4. hold a highly accountable class discussion about it. Often at the end, the outcome is revealed for comparison to the students prescriptions or predictions.  Nice. Tidy.  Print, repeat.

And print they did.  HBS alone has 17,000 case studies for sale!

But what if you took the whole concept up a notch?  What if the outcome was unknown because it simply hadn't happened yet?  Because the company was making the crucial decisions right now?  Today and tomorrow and throughout the course and, if the company survived, long after the students left the classroom.

And what if you had a front seat to the action? A reporter inside the company documenting what was happening week by week, interviewing executives and employees, and sending out updates once a week, including company documents, strategy, and financial forecasts?

And what if you could actually speak to the executives via video conference and ask them questions once a week?  And what if you were allowed to critique their business strategy using the same information they have available to them and provide recommendations for their consideration?

This is precisely the audacious experiment conceptualized and implemented by James Theroux at the Isenberg School of Management.  The results were phenomenal.  In the first implementation of this approach, 45% of  the students said they felt it was the most memorable or valuable course they had taken.

And was it good for the company?  Why would they air their dirty laundry in front of and answer questions from a bunch of students when they have urgent work to do and critical decisions to make?  Well, consider that the students identification of core weaknesses in the business model turned out to be accurate and their recommendations for action prophetic.

According to Theroux:

“Over time, and including [a] merger, the result has been to a large degree that [the students'] thoughts on positioning the company have come to fruition.”

Repeating this kind of success is difficult.  In a later iteration, a mere 25% of the students found it to be the most memorable or valuable course they had taken (I hate it when only a quarter of my students think my class was the best one ever!). The company in this case was slower moving, the technology field it was in was harder to understand, and the access to the leadership was, unexpectedly, more limited.

But the best things to do are way too often also the hardest things to do.  It is worth trying again and again to get a model like this right; to refine the selection criteria; to get the costs of engaging in the approach manageable (either by lowering execution costs or by scaling participation fees); to demonstrate benefits so compelling to both the company and the learning institution that the approach becomes accepted and routine and takes it place along side that other, once radical, approach that has been working since 1920.

Hats off to Dr. Theroux!

For more information on Real-Time Case Method, see:
Real-Time Case Method: Analysis of a Second Implementation. Theroux, James M.
Journal of Education for Business Jul/Aug 2009, Vol. 84 Issue 6, p367-373 7p; 2 Charts, 12 Graphs, 2009 

The Power of Learning Analytics, Targeted Intervention, and Mastery Learning at Scale

This is the first time I have ever reblogged, but the posting by Bror Saxberg, CLO of Kaplan, regarding applying learning analytics at scale at the University of Wisconsin at Milwaukee is worth repeating.  It is a powerful combination of large scale analytics, careful experimental design, targeted intervention,  semi-automated student motivation, and mastery learning. Make sure you jump to the original via the link below to see some very compelling charts representing their success.

By the way, I ran in to Bror at the Educause Learning Initiative conference, and he is more than ready to lead the revolution for the application of learning sciences to higher education, and he has enough energy for a dozen mere mortals to pull it off!  I am interested in connecting with others who share this passion as well. 

Bror writes:

At the recent Educause Learning Initiative meeting in Austin Texas, I came across some very interesting recent randomized control trial results from the University of Wisconsin at Milwaukee.  They’re testing a combination of mastery learning plus “amplified assistance” (data-driven suggestions for faculty about who to intervene with and how) in an introductory Psychology course, and, with thousands of students (!!) having run through controlled trials, they’re showing significant improvements in pass rates and long term retention.

via brorsblog.typepad.com

Adventure Learning: Real Time, Real People, Real Issues

Imagine teaching biology to a group of students in a typical classroom and beginning class this way:

And now we will chat with a scientist on an expedition in Greenland answering question we have about changes in the ecology due to warming climate trends.

Or imagine learning about sustainability while following a team of scientists and educators through Africa, speaking with local villagers about sustainable agricultural practices or the impact of globalization. These are both possible thanks to some incredibly forward thinking and fearless innovators at the University of Minnesota. They call this hybrid distance education approach, Adventure Learning.

To quote from their wikipedia article:

Adventure Learning (AL) ^[1] is a hybrid distance education approach...[that provides] students with opportunities to explore real-world issues through authentic learning experiences within collaborative learning environments, and is anchored in experiential and inquiry-based learning.^[2] The AL approach includes educational activities that work in conjunction with the authentic experiences of researchers in the field. For example, within an AL program, the curriculum, the travel experiences and observations of the researchers, and the online collaboration and interaction opportunities for participating learners are delivered synchronously so that learners are able to make connections between what is happening in the real world and their studies, and then reflect on those events and present potential solutions to issues that are raised.^[3]

The real world is happening 24/7 in every country on every continent on the planet. Why is it that when we enter a classroom setting, or even, unfortunately, when we enter most online courses, we shutter the windows as it were to the rest of the world. Why don't we invite that world in? Why don't we make the human connections that are possible now between what we are learning and the people that live those lessons every day?

Way back in the mid-90s, I ran an internet science camp for middle school kids. They were marginally engaged. Then one day we sent an invitation to an online group of scientists asking if they would chat with our class. To our great surprise, we had about 10 respond with an enthusiastic YES! I can still remember the day that our students were chatting live with scientists who were designing the Mars rovers or researching at Microsoft or DuPont. Our students were completely engaged, and everything we were trying to teach them about a future career in science was demonstrated in spades by these willing professionals reaching out to them with answers and advice.

How far can we take this approach? Imagine a language learning class following peers of the same age traveling through a country speaking the target language and engaging the target culture or just connecting with a class of local students. Imagine a science class connected to a team designing a solar car for the World Solar Challenge. Imagine an ancient history class following an excavation at Petra.

The possibilities are endless. Yes, it takes some extra effort. But, wow, isn't it worth it?

The Chocolate and Peanut Butter of Learning Analytics

On October 23rd, 2009, Adobe Systems, Inc. competed its acquisition of Omniture, Inc. An Adobe press release describes the benefits of this merger as follows:

The combination of the two companies will increase the value Adobe delivers to customers. For designers, developers, and online marketers, an integrated workflow — with optimization capabilities embedded in the creation tools — will streamline the creation and delivery of relevant content and applications. This optimization will enable advertisers and advertising agencies, publishers, and e-tailers to realize greater ROI from their digital media investments and improve their end users' experiences.

Beneficiaries: advertisers, advertising agencies, publishers, etailers
Benefit: greater ROI, better end user experience

What this press release does not say is that on that fateful day the chocolate of Omniture's web analytics tools dropped smack dab into the middle of the peanut butter of Adobe' s eLearning suite. Or it could have/should have/will if someone stops trying to flip up the most enticing Flash banner ad possible for a minute and thinks about the eLearning world. If the red and black eLearning folks will wander over to their new 1.8 billion dollar, lime green roommates and make a modest proposal, we could have a match as classic and enticing as the Reece's peanut butter cup.

Does Adobe have any idea that they are sitting on the biggest revolution in eLearning since the browser? Do they realize that they now have at their finger tips all of the tools necessary to dominate the eLearning space with the hottest, most completely integrated, most elegantly implemented learning analytics suite on the market? Who else has the power to build learning analytics straight into the most popular tools for elearning design and development? Who else has the statistical and number crunching guns to process and display massive amounts of learner data in slick, easy to use dashboards? Could there be a more obvious fit? Does anyone there realize that they could be the engine that powers a massive emerging industry?

Probably not. Why would the people focused on ROI for advertisers start scribbling on the back of napkins with people who are focused on ROI for eLearning ? Nothing against Adobe; big corporations just don't innovate this way very often. If they did, the press release might read as follows:

The combination of the two companies will increase the value Adobe delivers to learners everywhere. For instructional designers, eLearning developers, and online colleges and universities, an integrated workflow — with learning optimization and tracking capabilities embedded in the creation tools — will streamline the creation and delivery of customized learning content and experiences. This optimization will enable teachers, trainers, instructional designers, and training organizations as well as online educators to realize greater ROI from their digital training and teaching investments and improve their end users' experiences and, ultimately, the overall appeal, effectiveness and efficiency of their learning.

It would be a crying shame if this didn't happen. Imagine a world without Reece's peanut butter cups. It would be that bad.

Johnny Depp Kills Disney Pirates

Having just returned from Disneyland, I have been thinking about the updated Pirates of the Caribbean ride that now features Johnny Depp as Captain Jack Sparrow. While Disney surely added Capt. Jack to the ride to update it and increase audience engagement, in discussing it with my wife, it became clear to us that the addition actually reduces the narrative power and, thus, the overall engagement and enjoyment of the ride.

Timelessness

What makes good narrative great is when it is imbued with a sense of timelessness. While stories like Huckleberry Finn or To Kill a Mockingbird have a definite time and place, they don’t feel bound to that particular time. The characters feel fresh and real every time you read them, and the settings feel like they could exist right now in some parallel universe coexistent to our own. Before the addition of Jack Sparrow, the ride had this feeling to it. The riders filled in the details of the narrative, imagining a back story and a conclusion, projecting the motivation of each character and their ultimate fate, placing the events in an imaginary, but plausible, history. With the addition of Sparrow, all that is taken away. The narrative is forced (rather ungracefully in its repetition) upon you. You are now watching a version of a blockbuster movie, and you know the back story and the ending because it has been thrust upon you. The narrative has been appropriated, and you must comply.

Rider as a Character

Even worse is the loss of the inclusion of the riders as characters in the story. In the original ride, you become a part of the story. You are warned at the beginning that you are entering into danger, you then view the “cursed treasure,” and are then told the following:

“No fear hath ye of evil curses, says you. Ah... Properly warned, ye be, says I. Who knows when that evil curse will strike the greedy beholders of this bewitched treasure.

Perhaps you/ye knows too much. You've seen the cursed treasure. You know where it be hidden! Now pass through at your own risk. These be the last friendly words you'll here!”

You are now cursed and must face your fate. That curse brings you right into the middle of a fierce battle with cannon balls whizzing overhead, then through a burning, groaning, crumbing building, and finally between the muskets of drunken, dueling pirates. Happily, you survive all of these and leave with your secret knowledge of the treasure and your lives. All this is taken away from you in the new version. You simply watch as other characters play out other stories. The boat is little more than a floating theater seat. Pass the popcorn.

Implications for Instruction

Seeing or hearing a narrative is a powerful means of engagement. Even more powerful is direct participation in the narrative by taking on a role and/or being asked directly or indirectly to project the conclusion. People like Stephen Covey teach using great narratives. But they keep those narratives for themselves because they provide all the pieces and trot them out in each training masterfully and verbatim like an actor on a stage. While this is so much better than PowerPoint bullets, it is not as powerful as sharing the narrative with the audience, giving them a real stake in its outcome, and even trusting them to write the ending.

Unforgetable Learning: What Master Storytellers Can Teach Us About Designing Transformative Learning Experiences

This post is the story of the dissertation I didn't write, the book I would like to edit to make up for it, and how you can help.

Have you ever been channel surfing and came across a documentary about a topic that, until this pivotal moment, was of mild interest and found yourself, within 10 minutes, fully engrossed in every aspect and detail of say, the life cycle of a shrew, the mating rituals of the common slug or the travails of a nomadic Mongolian family whose camel has rejected her calf? Do you find yourself telling your co-workers the next morning with great enthusiasm about the way computer chips are manufactured or how FedEx sorts packages or how sea turtles migrate thousands of miles to the exact same beach on which they were born to lay their eggs having never visited it again in the intervening decades?

My dissertation was going to be a deep study of the structure of the most engaging, memorable documentaries designed for a general audience. I wanted to extract principles of how to engage and enthrall learners who only have a passing interest in a subject. It seemed to me that our field could learn a lot from those outside the field who had mastered this craft. Alas, I couldn't muster enough interest among my faculty and had to move on to another topic.

Master storytellers design their narrative to completely invest their audiences in their story; and audiences willingly give their rapt attention and full emotional involvement in return. I want to know how that is accomplished by the best storytellers in (at least) the following genres:

• Documentaries
• Feature Films
• Television Series
  
• Plays
• Musicals
• Dance Performances
• Poetry
• Short Stories
• Novels
• Video Games
• Role Playing Games
• Theme Park Rides

I believe much could be learned from a minute by minute analysis of the structure of some exemplars, by interviewing their creators, by interviewing their loyal audiences (or eavesdropping on their public online discussions), and by carefully studying those who experience these for the first time. I also believe that the narrative traditions, folk wisdom, and design tools of the individual genres could be mined for insight. For example, I did a minute-by-minute analysis of the NOVA episode on the Wright Brothers and found very clear and compelling structures designed to draw in and hold audiences with different levels of familiarly and interest in the topic.

I would love to serve as editor or co-editor on such a book, with chapters written by you or people you know who would be ideal candidates. I would be interested to hear from those who would like to participate or who know people who would like to submit a chapter or could recommend a publisher that would find such a book particularly interesting. Actually, I would just plain like to know what you think of the idea.

What do you think?

The Hidden Park: Great Concept, Cool Technology, Careful Narrative Design

Imagine a portable learning toolbox with these capabilities:

• GPS
  
• A digital camera
• An accelerometer

Now bring to it a conceptual framework:

• Alternative reality gaming
• Geochaching
  

What are the possibilities? What if you added a bit of story, a bit of fantasy, and loads of imagination?

You or I probably still wouldn't come up with The Hidden Park, but, luckily for us, the people at Bulpadok did. Here is a brief description from their press kit:

The adventure begins when the children receive a video call from a troll named Trutton, head of the Magical Wildlife Protection Association. Trutton explains that their park is in danger of being bulldozed by greedy developers. The kids must collect evidence to prove the existence of magical animals in their park.

The children navigate their way through the park by following a map that lets them know where the magical creatures live. Of course, Trutton’s map is magical – as they move past landmarks in the park the map tells them where to go next. The children must solve puzzles and riddles on their way to the next destination. Clues to the answers can be found on the signposts in the park.

Following Trutton’s directions, the children take photos of various landmarks. As if by magic, Trutton’s fantastical friends appear in the photos – sometimes right next to the children! The photos are stored in a gallery, so at the end of the day the children have an album of their adventure.

Very, very cool. So: Great concept. Smart, innovative people thinking outside of the box. Nifty technology. Contemporary aesthetic look and feel. That's what it takes, right? But what role did narrative design play? Narrative is clearly an element, but did it matter? Did they just hit the jackpot of a serendipitous cross over of cool technologies and a neat concept and then threw in a fun story as the frosting on the cake?

Oh, no. Not even close.

If you go to the blog of the creators of the game and look up the post titled Shaping the Story, you will see that they took the narrative element very, very seriously. These cutting edge game designers discuss Aristotle's classic three act structure, the narrative arc, and several modern storytelling theorists. Narrative design, it appears, was a central consideration of their game design, and, I would venture, central to the success of the product. These people thought long and hard about what the right narrative structure for the product should be, and have some very interesting thoughts on the mapping of a story arc to physical geography. An excerpt:

One of the really fun things about writing The Hidden Park was being able to physically map the narrative shape over the landscape. Many of the pathways that we plotted actually physically drew an arc through space. While this was satisfying at some theoretical level, it also became important for storytelling. We didn’t want people to double back over territory they had already covered and we didn’t want people to cross back over their path. The Hidden Park is a simple linear narrative and it was important for that to be reflected geographically. We wanted to create the feeling of physically moving forward through the story.

As GPS gaming evolves, it may be desirable to maintain a consistency between the shape of the narrative and its position in real space. Where a detective story leads the protagonist around in circles, a GPS mystery may literally lead the player back to where they started. Often in a story, a writer wants to revisit a particular theme and reveal something new to their audience. By requiring a player to physically return to a location, their path will trace the intricate folds and layers of sophisticated storytelling.

They go on to say that this mapping of physical space to the storytelling structure is not a hard and fast rule. But my point is to call attention to their careful consideration to the narrative structure of the product. They did their theoretical homework, and it pays off in the final product. Without a strong narrative, the product would have been gadgety cool and amusing. With the narrative, it becomes a compelling adventure that leads the player forward, as they state, "with direction and purpose."

So if we revisit the list of key features from their website, there seems to be a hidden element that didn't make the list:

• GPS
  
• A digital camera
• An accelerometer
• Alternative reality gaming
• Geochaching

and...

• ???

Go ahead, snap a photo with your iPhone and see what shows up...

Special Issue of Interactive Learning Environments Focuses on Narrative

I was recently had an article published as part of the December issue of the Interactive Learning Environments entitled "Designing video narratives to contextualize content for ESL learners: a design process case study." The issue was focused exclusively upon narrative and interactive learning environments and guest edited by Paul Brna and Rose Luckin. Below is the abstract of my article:

In this paper we discuss how the Brigham Young University Technology Assisted Language Learning Group (BYU TALL Group) develops video-based dramatic narratives to increase the amount of context we provide to English as a second language (ESL) learners. First, we discuss the problem of decontextualization in education, the contextualism alternative, and how narrative can provide crucial context. Next, using ESL instruction as a case study, we compare non-narrative video-based language models with narrative models and discuss some of the potential benefits of narrative models. We then discuss issues to consider when using narrative models, and outline a narrative-focused design and development process with particular attention to those aspects critical to creating narratives that are simultaneously pedagogically sound, aesthetically credible, and engaging for learners to watch.

"Smoking is [wags finger] not allowed on any delta flight."

Like a lot of people, sometime in the last couple of years, I was sitting on a Delta flight, bouncing out to the runway, when the heretofore totally ignorable safety video came on. But this time, it was different. Jazzy music. I looked up. Zippy editing. I kept watching. Amplified, stylized sound effects. Catchy. Then came the moment where all other safety videos got classified as boring and lame -- the finger wag, which you just simply have to see for yourself.

I laughed out loud. On it went with slick graphics, interesting camera angles and focus pulls, and even a post-production twinkle and audible "ding" on the life preserver demonstrator's smile.

This Delta safety video, in my opinion, is a great example of putting the engagement back into instructional design. While there has been a lot of talk about the attractiveness of the main actress (for the record, she is not an actress but an actual Delta flight attendant) as the reason people watch, I think it is more than that. Yes, she is striking, and that does increase engagement, but it is the sum of all the little touches mentioned above that raise the overall engagement level. One of those strategies (sound effect, focus change, unusual camera angle, etc.) is used about once every ten seconds or so. They are sprinkled throughout to (pleasantly) surprise you in every part of the video. If you doubt that this video is substantially different that other airline safety videos in terms of engagement, show me another safety video with over 1.2 million voluntary views on YouTube.

Now, we must remember that the content of the video did not substantially change from the previous safety video. In most cases, the exact same wording is used. The exact same action. It is all about the presentation, the pacing, the music, the little touches that draw the audience in. These "touches" are not touches at all. They are engagement strategies and should not be considered optional afterthoughts to the overall design. We tend to use pejoratives to describe them: window dressing or eye candy or Easter eggs. We think of the designers as being mischievous or even frivolous, of messing around after the serious work was done, by adding them. But they are the difference between whether people choose to watch the video or not. For many travelers, I would submit, they are the deciding factor in whether or not people read, sleep, or otherwise ignore the safety video or watch it.

As long as instructional design models continue to be derived from research in settings in which the audience is required to participate, our models will continue to exclude these pivotal factors in determining whether or not there is an audience at all. And it is a mistake to think that just because you have a bunch of people trapped in a classroom, or a training site, or an airplane, that you have an audience.

"Engagement [wags finger] is not optional in effective instructional design."

Web Analytics in Education

A few months ago, I saw a presentation by Clint Rogers on web analytics at BYU. Since then, I have been thinking a lot about the possibilities of web analytics for education. I even told my friends that I am pretty sure that whoever figures out how to do it right first is going to start an industry. I was pleased to see that Clint was teaching a seminar on that very topic this fall at BYU so I am sitting in on it even though I have almost no spare time right now. As a result, several upcoming posts will be related to this topic.

So what could we learn from web analytics that we don't already know and what could we do with that knowledge?

My brainstorm:

• We could know exactly who looked at what and for how long. We could know which of the 10 things we thought they absolutely had to read they actually did read (or at least left open on their browser) and for how long and then correlate that to their scores to see if they really did need to read those ten things or not.
  
• We could find out if the $5000 simulation we built gets more actual student face time than the $500 game.
  
• We could provide approach A 50% of the time and approach B 50% of the time and correlate to outcomes to see if one has better results.
  
• We could identify learners who are not logging in, or clicking randomly, or only doing the quizzes and intervene by notifying them automatically (but as if we are human) that we have noticed this pattern and we are concerned (a human would read the reply, of course).
  
• We could possbily identify profiles of people who are cheating.
  
• We could find out if online students really do cram the entire course in to the last three weeks of the semester and still get an A- on the final and reflect on how we feel about that.
• We could discover that you only need to skim this particular course to get a B-.
• We could discover that if you only read the intro and the summaries of each lesson you get a passing C.
• We could discover that those who do all the optional quizzes and pace themselves so that they complete three lessons a week get an A and then tell new students at the beginning of the course of this pattern for success in this particular course to help them invest in good study practices. And, if they fall off the wagon, we could remind them that their current, not so hot learning patterns correlate with a D for 90% of the students last semester that fell into this pattern and didn't change by October 1st. In fact, profiling the behavior of high performing students or of those who get off to a rough start and recover or of those who spend the least amount of time in the course but get the highest grades or, or, or..., I think, is one of the most interesting areas that could be investigated and could lead to a lot of good advice for others taking the course and entire course redesigns to make them more lean and mean and precisely helpful. Especially if we can profile the students entry characteristics and then correlate them to success patters for those specific characteristics.

  "Dear student, According to the survey and your past grades, you are very similar to 86 students who took this course in the last 2 years. These students also 'enjoyed working on their own' but 'felt that they learned slower than most' and had similar grades to you on the pre-requisite courses. Students with this profile were most successful in this course when they followed these study habits: yada yada However, most of these students were more inclined to follow these less effective patterns: yida yida. We have sophisticated tools that can produce a weekly report showing how close your study habits are to those of students with your profile who were sucessful in the past and warning if you fall into the less effective learning patterns common to students with your profile. Would you like us to send this report to you?"

None of this feels like TLC for the student, but I believe that the hard numbers and statistical patterns can be presented in a very human, non-threatening, helpful way that really will help students feel like the course designers/instructors know them and are there to help them and have this almost magical insight into how they can improve their performance in the course. Maybe not. But it is very much worth a try.

Fire that (Fictitious) Employee! Unanticipated Consequences of Using Narrative in Instructional Design

I went to an interesting presentation at my local chapter of ISPI by Andrew Wolff of PriceWaterhouseCoopers. He talked about how they had recently begun using the simplest, cheapest versions of narrative and humor in their training. For example, to help their people understand the technical side of one of their businesses, they show a series of photos with audio where a guy gets a call at the end of the week that he needs to do a report on the chipset the company is selling. He is about to ignore the request and go home when his cell phone starts to talk to him. They have still drawings of a little talking cellphone, that change every ten seconds or so in an "animation," and this cell phone has a cartoon-y character voice. The talking cellphone tells the guy about the importance of the chips inside it to the business's bottom line and takes him on a tour of the factory. Or they had a confidentiality training where a story plays out where a character makes simple mistakes that leads to a major breach of security for the company. These innovations were not very expensive and didn't take much longer than a vanilla course to produce. Among the effects, the ones that stood out to me the most were:

1. With no promotion whatsoever of the new course other than word of mouth, training completion timeframes for the company went from something like 90% in the last three days before the deadline for training completion to 90%+ in the first three days the course was available.

2. In the case of the security training, partners in the firm were calling the training department in the first few days after the training was released, trying to get the (fictional) character in the training fired for her negligence.

Now you tell me some other strategy that would have led to similar outcomes. And think of what the company stands to gain by shaving three months off of the amount of time it takes for all of their people to complete required training. And imagine the employees of your company talking to each other in the halls about the great confidentiality training they just completed and how you don't want to miss it. Sounds like some kind of training department fantasy. One that I think many of us would like to be in.

Where is Emotion, Engagement, and Aesthetics in the Learning Sciences?

I am looking at the index to the The Cambridge Handbook of the Learning Sciences. I am surprised to find:

• No entry for "engagement"
  
• No entry for "aesthetics"
  
• Only 2 pages under "emotions," one of which refers to this passage:
  

  We need a better understanding of the intertwining of affective, relational, and communicative aspects of learning interactions. How do emotional responses mediate learning, and how do they emerge from learning? (p.29)

  Good question!
  

• Only 2 pages under "narratives"
  
• Under "motivation," which has 21 sub-headings and 55 page references, there are a handful of possibly relevant subheadings: "Attention and motivation," with 1 page listed, "boredom and motivation," with 2 pages listed, "deep level engagement and motivation," 1 page listed, "emotions and motivation," 1 page listed.
  

Could someone clue me into to what a learning scientist might call "emotions," aesthetics," "engagement," and "narrative"? Could the field really have attended so little to these issues? I realize that the Handbook is hardly the entire corpus of the field, but I guess I was hoping to find a bit more than I did.

Planning for Engagement in Instructional Design

Pat Parrish had an engagement plan for his course. It looked* something like this:


He thought that students would generally start with lower levels of engagement, that that engagement would grow as they learned new material and completed assignments, that it would plateau in the middle of the course, and then rise to a climax near the end of the course when the applications of their learning became more apparent to them. Pat measured the engagement of each student throughout the course. Each line represents one student's reported engagement on a per module basis. This* is what he found:



Wow. Try following any one path through the chart. Now compare it to any other path on the chart. Explain why the two paths are different. Good luck.

So what's the point? Two points.

First point: Pat is way ahead of the curve. Pat actually had an engagement plan. He actually thought about how each part of the course would engage learners and to what extent he thought it would. He actually implemented learning activities to reflect his plan. Have any of the rest of us really tried that? Do we have a plan for engaging students in any systematic way? Do we have picture of the ideal engagement arc of our course in our heads? Or are we just focused on achieving learning objectives (somehow) or, worse, content coverage, and hope/assume that engagement will happen? Or are we resigned to the sad fact that learners choose to be engaged or not engaged, period, not my problem? Is this how a screenwriter, a playwright, or a music composer would think about their audience? I believe that while learners do have a choice to engage, we also have a choice of deciding how seriously we are going to try to reach out to them in engaging ways. How determined of a suitor of meaningful student engagement are we going to be?

Second point: Pat's students were all over the map. And we have no idea why (though I imagine that Pat has some guesses). Most of us have students like Pat. I would bet valuable property that 95% of us would find a similar, random looking set of curves if we tried the same experiment in our courses. We have no idea why they are or aren't engaged. We have no idea why in module 7, one student who had been averaging between a 6 and a 4 dropped to a 2 when six other students posted increasing engagement scores for the same module. This is the sort of thing that I feel like we really, really need to know. We should be able to read these patterns, perhaps not easily or perfectly, but we should at least have a sense of why these things are happening.

If we want to engage learners, and I do, we had better start finding ways to create and understand charts like Pat's. We are at the starting point of this kind of research (to my knowledge, if I am wrong, please let me know). The point when everything looks like random chaos. But it isn't random. There is a reason for every bend up or down on those curves. Let's go find out what is going on so we can design in engagement into our learning experiences. Yes, the learners have to choose, but let's give them every reason to choose to engage.

*Graphics used by permission; taken from an AECT 2007 presentation. Update: looks like Pat put the paper on his website.

Real-time Measure of Learner Engagement

I want to conduct this study:

Twenty college students (or learners of your choice) are given a means and a prompt (and a reward) to answer the following two questions every five minutes for all of their waking hours over the course of a day/week:

1. What are you doing right now (if different than your previous answer to this question)?
2. How engaged do you feel right now on a scale from 1 (I am bored to tears! Save me!) to 5 (Shhh! Go away, I am busy!)?

This would allow us to formulate some baseline data to see where learning experiences fall in the overall spectrum of a learner's environment in terms of engagement. I am guessing that most of them fall into the bottom twentieth percentile, most of the time. If so, that can't be good. This kind of study could also be instrumental in identifying those exceptional learning experiences that are maxing out the scales. Think we might want to look at those particular experiences a little more closely?

Or imagine this variation: You are the instructor of a course. Each student has a little engagement meter, asking to rate engagement on a scale from one to five every five minutes during your class time. You videotape the class. You synchronize the video with the data. You chart engagement across time. Where you see peaks (hopefully) and valleys (inevitably) you jump to that part of the tape to see what was/wasn't going on. How much do you think you could improve your course after just one session of this? After three? Five? Do you have the courage to have the data reported to you in real-time while teaching the class? (Would that even be a good idea?)

If you know of anyone anywhere who is doing anything along these lines, please let me know. I am aware of classroom clickers, but not aware of anyone using them to measure engagement throughout class time to create an engagement graph. And I have never heard of anyone trying to establish an "engagement baseline" for learners that compares their learning experiences to the rest of the daily experiences in their life.

In closing, could you please rate your level of engagement with this blog post on a scale from one to five? ;)

On Beyond ADDIE: Narrative, Aesthetics, & Learner Emotion

I cut my teeth in instructional design by designing a web-based air quality meteorology course. It all started when, one day, the director of my organization walked down the hall, stopping in at each office, asking, "Does anybody want to build a course for NOAA?" I said, "Sure!" and got to work.

At the time, I had no practical experience in the field to speak of. I had taken one class called "Introduction to Instructional Design." So I opened up my Principles of Instructional Design textbook from the class and started from page one. It was a great resource and got me up and running quickly. Between myself and two meteorology graduate students from NCSU, we produced the entire course that summer. To my knowledge, it was one of the first self-contained courses online and can still be found here (if you visit, please make sure you picture the state of the web in 1996 -- Hotmail was launched that summer and there was a whopping 342,081 websites online).

With time and experience, however, I have come to feel that the ADDIE-type approaches like the one in my Principles of Design book too often fail to account for the humanness of the learner. While you can use those methods to consider the humanity of your audience, you can also fulfill every prescribed step and entirely miss it.

What would I add? Let's start with three very powerful, very underutilized forces:

1. Narrative: Story has been used to bind people together in shared knowledge and understanding for thousands of years. It is arguably the first instructional strategy ever used to convey essential cultural knowledge to the rising generations. It's an essential aspect of virtually every culture on the planet. We are wired for narrative. We think in narrative, we speak in narrative, we even dream in narrative. We perceive our very existence as an unfolding narrative. We collectively pay billions of dollars to experience well-crafted (and not so well-crafted) narrative. Narrative design needs to be deeply understood and routinely practiced in our field.

How many instructional designers have even heard of the field of narratology? How many designers have studied the construction of a documentary, a screenplay, a dance performance, a musical composition? We are starting to scratch the surface with our recent attention to role-play scenarios and gaming, but have far, far to go.

2. Aesthetics: Human beings respond powerfully to aesthetic design. Every decision we make, like it or not, is mediated by our subjective perceptions. The "Bottomless Soup" study done by Brian Wansink, a recent winner of the IgNoble Prize for nutrition (and also has a book on the subject, Mindless Eating: Why We Eat More Than We Think) demonstrates this beautifully. And, of course, aesthetics don't only make us fat. They can relax us, orient us, inspire us, enliven us. Aesthetics are much more than the surface qualities of an object, but extend to encompass the richness of our experience, and the best applications of aesthetic design can embody and express layers of meaning in a profound, prereflective way. Patrick Parrish is starting the conversation in our field. This conversation needs to be accelerated and expanded.

3. Learner Emotion: Human beings feel, and what they feel influences their readiness to learn, their willingness to learn, how much they actually learn, and whether they will (ever) decide to learn about a particular topic again. As Russ Osguthorpe asks, "If they got an A in the class, and tell us that they never want to see that content again in their lives, have they really learned what we intended to teach them?" Emotions can work for or against learning. In order to account for emotion in our learning design, we need to know what learners are feeling before, during, and after learning experiences occur.

We have a whole science devoted to measuring learning before, during, and after learning experiences and, ostensibly, ways to intervene based on what is learned from these assessments. Where is the science and technique of measuring the learners' emotions? What are the best practices of how to intervene based in what is learned? What makes us think we can teach effectively if we only know what learners know and not how they feel? How they feel about learning this topic, how they feel about their ability to learn this topic, how they feel right now in this learning session during this learning activity? Engagement is both a cognitive and an emotional experience. Can you imagine a flow experience in a learning setting that was devoid of emotion? Can you imagine an overwhelmed, bored, distracted, or anxiety-filled learner maximizing their learning?

We teach human beings. Let's start designing human experiences as well as learning experiences.

Holistic Education vs. Holistic Learning vs. Holistic Learning Experiences

tsharvey (who also happens to be my brother-in-law) asked this question and I thought it was worth responding to in a post rather than just a comment.

tsharvey said:

I don't think I've ever encountered the term 'holistic learning experiences' —of course, I'm not in the instructional design field. While I can speculate what you might mean by holistic learning experiences (and how it might differ from ideas such as integrated learning environments), can you clarify or provide examples of such approaches?

My answer: There is a Holistic Education movement that is more expansive than I am referring to that you can read about here or at wikipedia. If you are interested, a colleague of mine recommends this book:
Holistic Education: An Analysis of Its Ideas and Nature (The Foundations of Holistic Education Series, V. 8)*
(*Amazon link -- I am an Amazon associate -- but you can also buy it here and I won't get a dime.)

There is also something called "holistic learning" by Patrick G. Love and Anne Goodsell Love. They define this as

the integration of intellectual, social, and emotional aspects of undergraduate student learning. (reference)

That is closer, since I am interested in all three aspects of learning. So what exactly do I mean by "holistic learning experiences"? What I am getting at is that, in the field of instructional design, most models require an analysis of the content that breaks it down into ever smaller pieces until you have all these little parts of knowledge. Then, many models say, you teach this piece this way and that piece that way and when you are done teaching all of the pieces, the learner will know what they need to know. This can become a fragmented, decontextualized kind of experience. Our university system is an analogy, where individual disciplines can get so focused on themselves they lose sight of the big picture and fail to collaborate with each other to research cross-disciplinary issues.

When I say holistic learning experience, I mean designing the experience with the whole as well as the parts in mind. What will the overall structure be? How will we make sure the parts are related to each other meaningfully? When is it better for the learner to experience the material in larger chunks (and, at times, with more ambiguity)? This is one of the reasons I am interested in narrative, because I believe that narrative design has to pay attention to both the whole and the parts at the same time to be successful, and I think my field can learn from that.

"Holistic learning experiences" is a term I made up. I am open to suggestions if a better one comes to mind.

Joseph

Learning as a Human Experience

I was trained as an instructional designer and have worked as one now for over a decade. As Brenda Bannan-Ritland said to me recently, "I honor that tradition." But I feel drawn to other disciplines that can inform it, unconventional design approaches that can improve it, and a new emphasis on learners as human beings, whose hearts are inseparably connected to their heads. When we learn, we feel, and not enough designers of learning experiences care enough about that. And it is worth caring about. It is worth designing for. It can make all the difference.

In this blog, I explore the many, many places outside the field (and a few inside) where designers have chosen to account for the human experience in a holistic way. I explore what it feels like to be a learner and how design can impact that experience. I explore how we can make learning rewarding to the human mind and enriching to the human spirit.

Please join me.

Joseph