The Chocolate and Peanut Butter of Learning Analytics

On October 23rd, 2009, Adobe Systems, Inc. competed its acquisition of Omniture, Inc. An Adobe press release describes the benefits of this merger as follows:

The combination of the two companies will increase the value Adobe delivers to customers. For designers, developers, and online marketers, an integrated workflow — with optimization capabilities embedded in the creation tools — will streamline the creation and delivery of relevant content and applications. This optimization will enable advertisers and advertising agencies, publishers, and e-tailers to realize greater ROI from their digital media investments and improve their end users' experiences.

Beneficiaries: advertisers, advertising agencies, publishers, etailers
Benefit: greater ROI, better end user experience

What this press release does not say is that on that fateful day the chocolate of Omniture's web analytics tools dropped smack dab into the middle of the peanut butter of Adobe' s eLearning suite. Or it could have/should have/will if someone stops trying to flip up the most enticing Flash banner ad possible for a minute and thinks about the eLearning world. If the red and black eLearning folks will wander over to their new 1.8 billion dollar, lime green roommates and make a modest proposal, we could have a match as classic and enticing as the Reece's peanut butter cup.

Does Adobe have any idea that they are sitting on the biggest revolution in eLearning since the browser? Do they realize that they now have at their finger tips all of the tools necessary to dominate the eLearning space with the hottest, most completely integrated, most elegantly implemented learning analytics suite on the market? Who else has the power to build learning analytics straight into the most popular tools for elearning design and development? Who else has the statistical and number crunching guns to process and display massive amounts of learner data in slick, easy to use dashboards? Could there be a more obvious fit? Does anyone there realize that they could be the engine that powers a massive emerging industry?

Probably not. Why would the people focused on ROI for advertisers start scribbling on the back of napkins with people who are focused on ROI for eLearning ? Nothing against Adobe; big corporations just don't innovate this way very often. If they did, the press release might read as follows:

The combination of the two companies will increase the value Adobe delivers to learners everywhere. For instructional designers, eLearning developers, and online colleges and universities, an integrated workflow — with learning optimization and tracking capabilities embedded in the creation tools — will streamline the creation and delivery of customized learning content and experiences. This optimization will enable teachers, trainers, instructional designers, and training organizations as well as online educators to realize greater ROI from their digital training and teaching investments and improve their end users' experiences and, ultimately, the overall appeal, effectiveness and efficiency of their learning.

It would be a crying shame if this didn't happen. Imagine a world without Reece's peanut butter cups. It would be that bad.

Johnny Depp Kills Disney Pirates

Having just returned from Disneyland, I have been thinking about the updated Pirates of the Caribbean ride that now features Johnny Depp as Captain Jack Sparrow. While Disney surely added Capt. Jack to the ride to update it and increase audience engagement, in discussing it with my wife, it became clear to us that the addition actually reduces the narrative power and, thus, the overall engagement and enjoyment of the ride.

Timelessness

What makes good narrative great is when it is imbued with a sense of timelessness. While stories like Huckleberry Finn or To Kill a Mockingbird have a definite time and place, they don’t feel bound to that particular time. The characters feel fresh and real every time you read them, and the settings feel like they could exist right now in some parallel universe coexistent to our own. Before the addition of Jack Sparrow, the ride had this feeling to it. The riders filled in the details of the narrative, imagining a back story and a conclusion, projecting the motivation of each character and their ultimate fate, placing the events in an imaginary, but plausible, history. With the addition of Sparrow, all that is taken away. The narrative is forced (rather ungracefully in its repetition) upon you. You are now watching a version of a blockbuster movie, and you know the back story and the ending because it has been thrust upon you. The narrative has been appropriated, and you must comply.

Rider as a Character

Even worse is the loss of the inclusion of the riders as characters in the story. In the original ride, you become a part of the story. You are warned at the beginning that you are entering into danger, you then view the “cursed treasure,” and are then told the following:

“No fear hath ye of evil curses, says you. Ah... Properly warned, ye be, says I. Who knows when that evil curse will strike the greedy beholders of this bewitched treasure.

Perhaps you/ye knows too much. You've seen the cursed treasure. You know where it be hidden! Now pass through at your own risk. These be the last friendly words you'll here!”

You are now cursed and must face your fate. That curse brings you right into the middle of a fierce battle with cannon balls whizzing overhead, then through a burning, groaning, crumbing building, and finally between the muskets of drunken, dueling pirates. Happily, you survive all of these and leave with your secret knowledge of the treasure and your lives. All this is taken away from you in the new version. You simply watch as other characters play out other stories. The boat is little more than a floating theater seat. Pass the popcorn.

Implications for Instruction

Seeing or hearing a narrative is a powerful means of engagement. Even more powerful is direct participation in the narrative by taking on a role and/or being asked directly or indirectly to project the conclusion. People like Stephen Covey teach using great narratives. But they keep those narratives for themselves because they provide all the pieces and trot them out in each training masterfully and verbatim like an actor on a stage. While this is so much better than PowerPoint bullets, it is not as powerful as sharing the narrative with the audience, giving them a real stake in its outcome, and even trusting them to write the ending.

Unforgetable Learning: What Master Storytellers Can Teach Us About Designing Transformative Learning Experiences

This post is the story of the dissertation I didn't write, the book I would like to edit to make up for it, and how you can help.

Have you ever been channel surfing and came across a documentary about a topic that, until this pivotal moment, was of mild interest and found yourself, within 10 minutes, fully engrossed in every aspect and detail of say, the life cycle of a shrew, the mating rituals of the common slug or the travails of a nomadic Mongolian family whose camel has rejected her calf? Do you find yourself telling your co-workers the next morning with great enthusiasm about the way computer chips are manufactured or how FedEx sorts packages or how sea turtles migrate thousands of miles to the exact same beach on which they were born to lay their eggs having never visited it again in the intervening decades?

My dissertation was going to be a deep study of the structure of the most engaging, memorable documentaries designed for a general audience. I wanted to extract principles of how to engage and enthrall learners who only have a passing interest in a subject. It seemed to me that our field could learn a lot from those outside the field who had mastered this craft. Alas, I couldn't muster enough interest among my faculty and had to move on to another topic.

Master storytellers design their narrative to completely invest their audiences in their story; and audiences willingly give their rapt attention and full emotional involvement in return. I want to know how that is accomplished by the best storytellers in (at least) the following genres:

• Documentaries
• Feature Films
• Television Series
  
• Plays
• Musicals
• Dance Performances
• Poetry
• Short Stories
• Novels
• Video Games
• Role Playing Games
• Theme Park Rides

I believe much could be learned from a minute by minute analysis of the structure of some exemplars, by interviewing their creators, by interviewing their loyal audiences (or eavesdropping on their public online discussions), and by carefully studying those who experience these for the first time. I also believe that the narrative traditions, folk wisdom, and design tools of the individual genres could be mined for insight. For example, I did a minute-by-minute analysis of the NOVA episode on the Wright Brothers and found very clear and compelling structures designed to draw in and hold audiences with different levels of familiarly and interest in the topic.

I would love to serve as editor or co-editor on such a book, with chapters written by you or people you know who would be ideal candidates. I would be interested to hear from those who would like to participate or who know people who would like to submit a chapter or could recommend a publisher that would find such a book particularly interesting. Actually, I would just plain like to know what you think of the idea.

What do you think?

The Hidden Park: Great Concept, Cool Technology, Careful Narrative Design

Imagine a portable learning toolbox with these capabilities:

• GPS
  
• A digital camera
• An accelerometer

Now bring to it a conceptual framework:

• Alternative reality gaming
• Geochaching
  

What are the possibilities? What if you added a bit of story, a bit of fantasy, and loads of imagination?

You or I probably still wouldn't come up with The Hidden Park, but, luckily for us, the people at Bulpadok did. Here is a brief description from their press kit:

The adventure begins when the children receive a video call from a troll named Trutton, head of the Magical Wildlife Protection Association. Trutton explains that their park is in danger of being bulldozed by greedy developers. The kids must collect evidence to prove the existence of magical animals in their park.

The children navigate their way through the park by following a map that lets them know where the magical creatures live. Of course, Trutton’s map is magical – as they move past landmarks in the park the map tells them where to go next. The children must solve puzzles and riddles on their way to the next destination. Clues to the answers can be found on the signposts in the park.

Following Trutton’s directions, the children take photos of various landmarks. As if by magic, Trutton’s fantastical friends appear in the photos – sometimes right next to the children! The photos are stored in a gallery, so at the end of the day the children have an album of their adventure.

Very, very cool. So: Great concept. Smart, innovative people thinking outside of the box. Nifty technology. Contemporary aesthetic look and feel. That's what it takes, right? But what role did narrative design play? Narrative is clearly an element, but did it matter? Did they just hit the jackpot of a serendipitous cross over of cool technologies and a neat concept and then threw in a fun story as the frosting on the cake?

Oh, no. Not even close.

If you go to the blog of the creators of the game and look up the post titled Shaping the Story, you will see that they took the narrative element very, very seriously. These cutting edge game designers discuss Aristotle's classic three act structure, the narrative arc, and several modern storytelling theorists. Narrative design, it appears, was a central consideration of their game design, and, I would venture, central to the success of the product. These people thought long and hard about what the right narrative structure for the product should be, and have some very interesting thoughts on the mapping of a story arc to physical geography. An excerpt:

One of the really fun things about writing The Hidden Park was being able to physically map the narrative shape over the landscape. Many of the pathways that we plotted actually physically drew an arc through space. While this was satisfying at some theoretical level, it also became important for storytelling. We didn’t want people to double back over territory they had already covered and we didn’t want people to cross back over their path. The Hidden Park is a simple linear narrative and it was important for that to be reflected geographically. We wanted to create the feeling of physically moving forward through the story.

As GPS gaming evolves, it may be desirable to maintain a consistency between the shape of the narrative and its position in real space. Where a detective story leads the protagonist around in circles, a GPS mystery may literally lead the player back to where they started. Often in a story, a writer wants to revisit a particular theme and reveal something new to their audience. By requiring a player to physically return to a location, their path will trace the intricate folds and layers of sophisticated storytelling.

They go on to say that this mapping of physical space to the storytelling structure is not a hard and fast rule. But my point is to call attention to their careful consideration to the narrative structure of the product. They did their theoretical homework, and it pays off in the final product. Without a strong narrative, the product would have been gadgety cool and amusing. With the narrative, it becomes a compelling adventure that leads the player forward, as they state, "with direction and purpose."

So if we revisit the list of key features from their website, there seems to be a hidden element that didn't make the list:

• GPS
  
• A digital camera
• An accelerometer
• Alternative reality gaming
• Geochaching

and...

• ???

Go ahead, snap a photo with your iPhone and see what shows up...

Special Issue of Interactive Learning Environments Focuses on Narrative

I was recently had an article published as part of the December issue of the Interactive Learning Environments entitled "Designing video narratives to contextualize content for ESL learners: a design process case study." The issue was focused exclusively upon narrative and interactive learning environments and guest edited by Paul Brna and Rose Luckin. Below is the abstract of my article:

In this paper we discuss how the Brigham Young University Technology Assisted Language Learning Group (BYU TALL Group) develops video-based dramatic narratives to increase the amount of context we provide to English as a second language (ESL) learners. First, we discuss the problem of decontextualization in education, the contextualism alternative, and how narrative can provide crucial context. Next, using ESL instruction as a case study, we compare non-narrative video-based language models with narrative models and discuss some of the potential benefits of narrative models. We then discuss issues to consider when using narrative models, and outline a narrative-focused design and development process with particular attention to those aspects critical to creating narratives that are simultaneously pedagogically sound, aesthetically credible, and engaging for learners to watch.

"Smoking is [wags finger] not allowed on any delta flight."

Like a lot of people, sometime in the last couple of years, I was sitting on a Delta flight, bouncing out to the runway, when the heretofore totally ignorable safety video came on. But this time, it was different. Jazzy music. I looked up. Zippy editing. I kept watching. Amplified, stylized sound effects. Catchy. Then came the moment where all other safety videos got classified as boring and lame -- the finger wag, which you just simply have to see for yourself.

I laughed out loud. On it went with slick graphics, interesting camera angles and focus pulls, and even a post-production twinkle and audible "ding" on the life preserver demonstrator's smile.

This Delta safety video, in my opinion, is a great example of putting the engagement back into instructional design. While there has been a lot of talk about the attractiveness of the main actress (for the record, she is not an actress but an actual Delta flight attendant) as the reason people watch, I think it is more than that. Yes, she is striking, and that does increase engagement, but it is the sum of all the little touches mentioned above that raise the overall engagement level. One of those strategies (sound effect, focus change, unusual camera angle, etc.) is used about once every ten seconds or so. They are sprinkled throughout to (pleasantly) surprise you in every part of the video. If you doubt that this video is substantially different that other airline safety videos in terms of engagement, show me another safety video with over 1.2 million voluntary views on YouTube.

Now, we must remember that the content of the video did not substantially change from the previous safety video. In most cases, the exact same wording is used. The exact same action. It is all about the presentation, the pacing, the music, the little touches that draw the audience in. These "touches" are not touches at all. They are engagement strategies and should not be considered optional afterthoughts to the overall design. We tend to use pejoratives to describe them: window dressing or eye candy or Easter eggs. We think of the designers as being mischievous or even frivolous, of messing around after the serious work was done, by adding them. But they are the difference between whether people choose to watch the video or not. For many travelers, I would submit, they are the deciding factor in whether or not people read, sleep, or otherwise ignore the safety video or watch it.

As long as instructional design models continue to be derived from research in settings in which the audience is required to participate, our models will continue to exclude these pivotal factors in determining whether or not there is an audience at all. And it is a mistake to think that just because you have a bunch of people trapped in a classroom, or a training site, or an airplane, that you have an audience.

"Engagement [wags finger] is not optional in effective instructional design."

Web Analytics in Education

A few months ago, I saw a presentation by Clint Rogers on web analytics at BYU. Since then, I have been thinking a lot about the possibilities of web analytics for education. I even told my friends that I am pretty sure that whoever figures out how to do it right first is going to start an industry. I was pleased to see that Clint was teaching a seminar on that very topic this fall at BYU so I am sitting in on it even though I have almost no spare time right now. As a result, several upcoming posts will be related to this topic.

So what could we learn from web analytics that we don't already know and what could we do with that knowledge?

My brainstorm:

• We could know exactly who looked at what and for how long. We could know which of the 10 things we thought they absolutely had to read they actually did read (or at least left open on their browser) and for how long and then correlate that to their scores to see if they really did need to read those ten things or not.
  
• We could find out if the $5000 simulation we built gets more actual student face time than the $500 game.
  
• We could provide approach A 50% of the time and approach B 50% of the time and correlate to outcomes to see if one has better results.
  
• We could identify learners who are not logging in, or clicking randomly, or only doing the quizzes and intervene by notifying them automatically (but as if we are human) that we have noticed this pattern and we are concerned (a human would read the reply, of course).
  
• We could possbily identify profiles of people who are cheating.
  
• We could find out if online students really do cram the entire course in to the last three weeks of the semester and still get an A- on the final and reflect on how we feel about that.
• We could discover that you only need to skim this particular course to get a B-.
• We could discover that if you only read the intro and the summaries of each lesson you get a passing C.
• We could discover that those who do all the optional quizzes and pace themselves so that they complete three lessons a week get an A and then tell new students at the beginning of the course of this pattern for success in this particular course to help them invest in good study practices. And, if they fall off the wagon, we could remind them that their current, not so hot learning patterns correlate with a D for 90% of the students last semester that fell into this pattern and didn't change by October 1st. In fact, profiling the behavior of high performing students or of those who get off to a rough start and recover or of those who spend the least amount of time in the course but get the highest grades or, or, or..., I think, is one of the most interesting areas that could be investigated and could lead to a lot of good advice for others taking the course and entire course redesigns to make them more lean and mean and precisely helpful. Especially if we can profile the students entry characteristics and then correlate them to success patters for those specific characteristics.

  "Dear student, According to the survey and your past grades, you are very similar to 86 students who took this course in the last 2 years. These students also 'enjoyed working on their own' but 'felt that they learned slower than most' and had similar grades to you on the pre-requisite courses. Students with this profile were most successful in this course when they followed these study habits: yada yada However, most of these students were more inclined to follow these less effective patterns: yida yida. We have sophisticated tools that can produce a weekly report showing how close your study habits are to those of students with your profile who were sucessful in the past and warning if you fall into the less effective learning patterns common to students with your profile. Would you like us to send this report to you?"

None of this feels like TLC for the student, but I believe that the hard numbers and statistical patterns can be presented in a very human, non-threatening, helpful way that really will help students feel like the course designers/instructors know them and are there to help them and have this almost magical insight into how they can improve their performance in the course. Maybe not. But it is very much worth a try.