In this paper we discuss how the Brigham Young University Technology Assisted Language Learning Group (BYU TALL Group) develops video-based dramatic narratives to increase the amount of context we provide to English as a second language (ESL) learners. First, we discuss the problem of decontextualization in education, the contextualism alternative, and how narrative can provide crucial context. Next, using ESL instruction as a case study, we compare non-narrative video-based language models with narrative models and discuss some of the potential benefits of narrative models. We then discuss issues to consider when using narrative models, and outline a narrative-focused design and development process with particular attention to those aspects critical to creating narratives that are simultaneously pedagogically sound, aesthetically credible, and engaging for learners to watch.
Wednesday, May 20, 2009
Special Issue of Interactive Learning Environments Focuses on Narrative
I was recently had an article published as part of the December issue of the Interactive Learning Environments entitled "Designing video narratives to contextualize content for ESL learners: a design process case study." The issue was focused exclusively upon narrative and interactive learning environments and guest edited by Paul Brna and Rose Luckin. Below is the abstract of my article:
Friday, March 27, 2009
"Smoking is [wags finger] not allowed on any delta flight."
Like a lot of people, sometime in the last couple of years, I was sitting on a Delta flight, bouncing out to the runway, when the heretofore totally ignorable safety video came on. But this time, it was different. Jazzy music. I looked up. Zippy editing. I kept watching. Amplified, stylized sound effects. Catchy. Then came the moment where all other safety videos got classified as boring and lame -- the finger wag, which you just simply have to see for yourself.
I laughed out loud. On it went with slick graphics, interesting camera angles and focus pulls, and even a post-production twinkle and audible "ding" on the life preserver demonstrator's smile.
This Delta safety video, in my opinion, is a great example of putting the engagement back into instructional design. While there has been a lot of talk about the attractiveness of the main actress (for the record, she is not an actress but an actual Delta flight attendant) as the reason people watch, I think it is more than that. Yes, she is striking, and that does increase engagement, but it is the sum of all the little touches mentioned above that raise the overall engagement level. One of those strategies (sound effect, focus change, unusual camera angle, etc.) is used about once every ten seconds or so. They are sprinkled throughout to (pleasantly) surprise you in every part of the video. If you doubt that this video is substantially different that other airline safety videos in terms of engagement, show me another safety video with over 1.2 million voluntary views on YouTube.
Now, we must remember that the content of the video did not substantially change from the previous safety video. In most cases, the exact same wording is used. The exact same action. It is all about the presentation, the pacing, the music, the little touches that draw the audience in. These "touches" are not touches at all. They are engagement strategies and should not be considered optional afterthoughts to the overall design. We tend to use pejoratives to describe them: window dressing or eye candy or Easter eggs. We think of the designers as being mischievous or even frivolous, of messing around after the serious work was done, by adding them. But they are the difference between whether people choose to watch the video or not. For many travelers, I would submit, they are the deciding factor in whether or not people read, sleep, or otherwise ignore the safety video or watch it.
As long as instructional design models continue to be derived from research in settings in which the audience is required to participate, our models will continue to exclude these pivotal factors in determining whether or not there is an audience at all. And it is a mistake to think that just because you have a bunch of people trapped in a classroom, or a training site, or an airplane, that you have an audience.
I laughed out loud. On it went with slick graphics, interesting camera angles and focus pulls, and even a post-production twinkle and audible "ding" on the life preserver demonstrator's smile.This Delta safety video, in my opinion, is a great example of putting the engagement back into instructional design. While there has been a lot of talk about the attractiveness of the main actress (for the record, she is not an actress but an actual Delta flight attendant) as the reason people watch, I think it is more than that. Yes, she is striking, and that does increase engagement, but it is the sum of all the little touches mentioned above that raise the overall engagement level. One of those strategies (sound effect, focus change, unusual camera angle, etc.) is used about once every ten seconds or so. They are sprinkled throughout to (pleasantly) surprise you in every part of the video. If you doubt that this video is substantially different that other airline safety videos in terms of engagement, show me another safety video with over 1.2 million voluntary views on YouTube.
Now, we must remember that the content of the video did not substantially change from the previous safety video. In most cases, the exact same wording is used. The exact same action. It is all about the presentation, the pacing, the music, the little touches that draw the audience in. These "touches" are not touches at all. They are engagement strategies and should not be considered optional afterthoughts to the overall design. We tend to use pejoratives to describe them: window dressing or eye candy or Easter eggs. We think of the designers as being mischievous or even frivolous, of messing around after the serious work was done, by adding them. But they are the difference between whether people choose to watch the video or not. For many travelers, I would submit, they are the deciding factor in whether or not people read, sleep, or otherwise ignore the safety video or watch it.
As long as instructional design models continue to be derived from research in settings in which the audience is required to participate, our models will continue to exclude these pivotal factors in determining whether or not there is an audience at all. And it is a mistake to think that just because you have a bunch of people trapped in a classroom, or a training site, or an airplane, that you have an audience.
"Engagement [wags finger] is not optional in effective instructional design."
Tuesday, September 9, 2008
Web Analytics in Education
A few months ago, I saw a presentation by Clint Rogers on web analytics at BYU. Since then, I have been thinking a lot about the possibilities of web analytics for education. I even told my friends that I am pretty sure that whoever figures out how to do it right first is going to start an industry. I was pleased to see that Clint was teaching a seminar on that very topic this fall at BYU so I am sitting in on it even though I have almost no spare time right now. As a result, several upcoming posts will be related to this topic.
So what could we learn from web analytics that we don't already know and what could we do with that knowledge?
My brainstorm:
So what could we learn from web analytics that we don't already know and what could we do with that knowledge?
My brainstorm:
- We could know exactly who looked at what and for how long. We could know which of the 10 things we thought they absolutely had to read they actually did read (or at least left open on their browser) and for how long and then correlate that to their scores to see if they really did need to read those ten things or not.
- We could find out if the $5000 simulation we built gets more actual student face time than the $500 game.
- We could provide approach A 50% of the time and approach B 50% of the time and correlate to outcomes to see if one has better results.
- We could identify learners who are not logging in, or clicking randomly, or only doing the quizzes and intervene by notifying them automatically (but as if we are human) that we have noticed this pattern and we are concerned (a human would read the reply, of course).
- We could possbily identify profiles of people who are cheating.
- We could find out if online students really do cram the entire course in to the last three weeks of the semester and still get an A- on the final and reflect on how we feel about that.
- We could discover that you only need to skim this particular course to get a B-.
- We could discover that if you only read the intro and the summaries of each lesson you get a passing C.
- We could discover that those who do all the optional quizzes and pace themselves so that they complete three lessons a week get an A and then tell new students at the beginning of the course of this pattern for success in this particular course to help them invest in good study practices. And, if they fall off the wagon, we could remind them that their current, not so hot learning patterns correlate with a D for 90% of the students last semester that fell into this pattern and didn't change by October 1st. In fact, profiling the behavior of high performing students or of those who get off to a rough start and recover or of those who spend the least amount of time in the course but get the highest grades or, or, or..., I think, is one of the most interesting areas that could be investigated and could lead to a lot of good advice for others taking the course and entire course redesigns to make them more lean and mean and precisely helpful. Especially if we can profile the students entry characteristics and then correlate them to success patters for those specific characteristics.
"Dear student, According to the survey and your past grades, you are very similar to 86 students who took this course in the last 2 years. These students also 'enjoyed working on their own' but 'felt that they learned slower than most' and had similar grades to you on the pre-requisite courses. Students with this profile were most successful in this course when they followed these study habits: yada yada However, most of these students were more inclined to follow these less effective patterns: yida yida. We have sophisticated tools that can produce a weekly report showing how close your study habits are to those of students with your profile who were sucessful in the past and warning if you fall into the less effective learning patterns common to students with your profile. Would you like us to send this report to you?"
Wednesday, April 9, 2008
Fire that (Fictitious) Employee! Unanticipated Consequences of Using Narrative in Instructional Design
I went to an interesting presentation at my local chapter of ISPI by Andrew Wolff of PriceWaterhouseCoopers. He talked about how they had recently begun using the simplest, cheapest versions of narrative and humor in their training. For example, to help their people understand the technical side of one of their businesses, they show a series of photos with audio where a guy gets a call at the end of the week that he needs to do a report on the chipset the company is selling. He is about to ignore the request and go home when his cell phone starts to talk to him. They have still drawings of a little talking cellphone, that change every ten seconds or so in an "animation," and this cell phone has a cartoon-y character voice. The talking cellphone tells the guy about the importance of the chips inside it to the business's bottom line and takes him on a tour of the factory. Or they had a confidentiality training where a story plays out where a character makes simple mistakes that leads to a major breach of security for the company. These innovations were not very expensive and didn't take much longer than a vanilla course to produce. Among the effects, the ones that stood out to me the most were:
1. With no promotion whatsoever of the new course other than word of mouth, training completion timeframes for the company went from something like 90% in the last three days before the deadline for training completion to 90%+ in the first three days the course was available.
2. In the case of the security training, partners in the firm were calling the training department in the first few days after the training was released, trying to get the (fictional) character in the training fired for her negligence.
Now you tell me some other strategy that would have led to similar outcomes. And think of what the company stands to gain by shaving three months off of the amount of time it takes for all of their people to complete required training. And imagine the employees of your company talking to each other in the halls about the great confidentiality training they just completed and how you don't want to miss it. Sounds like some kind of training department fantasy. One that I think many of us would like to be in.
1. With no promotion whatsoever of the new course other than word of mouth, training completion timeframes for the company went from something like 90% in the last three days before the deadline for training completion to 90%+ in the first three days the course was available.
2. In the case of the security training, partners in the firm were calling the training department in the first few days after the training was released, trying to get the (fictional) character in the training fired for her negligence.
Now you tell me some other strategy that would have led to similar outcomes. And think of what the company stands to gain by shaving three months off of the amount of time it takes for all of their people to complete required training. And imagine the employees of your company talking to each other in the halls about the great confidentiality training they just completed and how you don't want to miss it. Sounds like some kind of training department fantasy. One that I think many of us would like to be in.
Wednesday, April 2, 2008
Where is Emotion, Engagement, and Aesthetics in the Learning Sciences?
I am looking at the index to the The Cambridge Handbook of the Learning Sciences. I am surprised to find:
. I am surprised to find:- No entry for "engagement"
- No entry for "aesthetics"
- Only 2 pages under "emotions," one of which refers to this passage:
We need a better understanding of the intertwining of affective, relational, and communicative aspects of learning interactions. How do emotional responses mediate learning, and how do they emerge from learning? (p.29)
Good question!
- Only 2 pages under "narratives"
- Under "motivation," which has 21 sub-headings and 55 page references, there are a handful of possibly relevant subheadings: "Attention and motivation," with 1 page listed, "boredom and motivation," with 2 pages listed, "deep level engagement and motivation," 1 page listed, "emotions and motivation," 1 page listed.
Thursday, March 27, 2008
Planning for Engagement in Instructional Design
Pat Parrish had an engagement plan for his course. It looked* something like this:
He thought that students would generally start with lower levels of engagement, that that engagement would grow as they learned new material and completed assignments, that it would plateau in the middle of the course, and then rise to a climax near the end of the course when the applications of their learning became more apparent to them. Pat measured the engagement of each student throughout the course. Each line represents one student's reported engagement on a per module basis. This* is what he found:
Wow. Try following any one path through the chart. Now compare it to any other path on the chart. Explain why the two paths are different. Good luck.
So what's the point? Two points.
First point: Pat is way ahead of the curve. Pat actually had an engagement plan. He actually thought about how each part of the course would engage learners and to what extent he thought it would. He actually implemented learning activities to reflect his plan. Have any of the rest of us really tried that? Do we have a plan for engaging students in any systematic way? Do we have picture of the ideal engagement arc of our course in our heads? Or are we just focused on achieving learning objectives (somehow) or, worse, content coverage, and hope/assume that engagement will happen? Or are we resigned to the sad fact that learners choose to be engaged or not engaged, period, not my problem? Is this how a screenwriter, a playwright, or a music composer would think about their audience? I believe that while learners do have a choice to engage, we also have a choice of deciding how seriously we are going to try to reach out to them in engaging ways. How determined of a suitor of meaningful student engagement are we going to be?
Second point: Pat's students were all over the map. And we have no idea why (though I imagine that Pat has some guesses). Most of us have students like Pat. I would bet valuable property that 95% of us would find a similar, random looking set of curves if we tried the same experiment in our courses. We have no idea why they are or aren't engaged. We have no idea why in module 7, one student who had been averaging between a 6 and a 4 dropped to a 2 when six other students posted increasing engagement scores for the same module. This is the sort of thing that I feel like we really, really need to know. We should be able to read these patterns, perhaps not easily or perfectly, but we should at least have a sense of why these things are happening.
If we want to engage learners, and I do, we had better start finding ways to create and understand charts like Pat's. We are at the starting point of this kind of research (to my knowledge, if I am wrong, please let me know). The point when everything looks like random chaos. But it isn't random. There is a reason for every bend up or down on those curves. Let's go find out what is going on so we can design in engagement into our learning experiences. Yes, the learners have to choose, but let's give them every reason to choose to engage.
*Graphics used by permission; taken from an AECT 2007 presentation. Update: looks like Pat put the paper on his website.
He thought that students would generally start with lower levels of engagement, that that engagement would grow as they learned new material and completed assignments, that it would plateau in the middle of the course, and then rise to a climax near the end of the course when the applications of their learning became more apparent to them. Pat measured the engagement of each student throughout the course. Each line represents one student's reported engagement on a per module basis. This* is what he found:
Wow. Try following any one path through the chart. Now compare it to any other path on the chart. Explain why the two paths are different. Good luck.
So what's the point? Two points.
First point: Pat is way ahead of the curve. Pat actually had an engagement plan. He actually thought about how each part of the course would engage learners and to what extent he thought it would. He actually implemented learning activities to reflect his plan. Have any of the rest of us really tried that? Do we have a plan for engaging students in any systematic way? Do we have picture of the ideal engagement arc of our course in our heads? Or are we just focused on achieving learning objectives (somehow) or, worse, content coverage, and hope/assume that engagement will happen? Or are we resigned to the sad fact that learners choose to be engaged or not engaged, period, not my problem? Is this how a screenwriter, a playwright, or a music composer would think about their audience? I believe that while learners do have a choice to engage, we also have a choice of deciding how seriously we are going to try to reach out to them in engaging ways. How determined of a suitor of meaningful student engagement are we going to be?
Second point: Pat's students were all over the map. And we have no idea why (though I imagine that Pat has some guesses). Most of us have students like Pat. I would bet valuable property that 95% of us would find a similar, random looking set of curves if we tried the same experiment in our courses. We have no idea why they are or aren't engaged. We have no idea why in module 7, one student who had been averaging between a 6 and a 4 dropped to a 2 when six other students posted increasing engagement scores for the same module. This is the sort of thing that I feel like we really, really need to know. We should be able to read these patterns, perhaps not easily or perfectly, but we should at least have a sense of why these things are happening.
If we want to engage learners, and I do, we had better start finding ways to create and understand charts like Pat's. We are at the starting point of this kind of research (to my knowledge, if I am wrong, please let me know). The point when everything looks like random chaos. But it isn't random. There is a reason for every bend up or down on those curves. Let's go find out what is going on so we can design in engagement into our learning experiences. Yes, the learners have to choose, but let's give them every reason to choose to engage.
*Graphics used by permission; taken from an AECT 2007 presentation. Update: looks like Pat put the paper on his website.
Thursday, March 20, 2008
Real-time Measure of Learner Engagement
I want to conduct this study:
Twenty college students (or learners of your choice) are given a means and a prompt (and a reward) to answer the following two questions every five minutes for all of their waking hours over the course of a day/week:
1. What are you doing right now (if different than your previous answer to this question)?
2. How engaged do you feel right now on a scale from 1 (I am bored to tears! Save me!) to 5 (Shhh! Go away, I am busy!)?
This would allow us to formulate some baseline data to see where learning experiences fall in the overall spectrum of a learner's environment in terms of engagement. I am guessing that most of them fall into the bottom twentieth percentile, most of the time. If so, that can't be good. This kind of study could also be instrumental in identifying those exceptional learning experiences that are maxing out the scales. Think we might want to look at those particular experiences a little more closely?
Or imagine this variation: You are the instructor of a course. Each student has a little engagement meter, asking to rate engagement on a scale from one to five every five minutes during your class time. You videotape the class. You synchronize the video with the data. You chart engagement across time. Where you see peaks (hopefully) and valleys (inevitably) you jump to that part of the tape to see what was/wasn't going on. How much do you think you could improve your course after just one session of this? After three? Five? Do you have the courage to have the data reported to you in real-time while teaching the class? (Would that even be a good idea?)
If you know of anyone anywhere who is doing anything along these lines, please let me know. I am aware of classroom clickers, but not aware of anyone using them to measure engagement throughout class time to create an engagement graph. And I have never heard of anyone trying to establish an "engagement baseline" for learners that compares their learning experiences to the rest of the daily experiences in their life.
In closing, could you please rate your level of engagement with this blog post on a scale from one to five? ;)
Twenty college students (or learners of your choice) are given a means and a prompt (and a reward) to answer the following two questions every five minutes for all of their waking hours over the course of a day/week:
1. What are you doing right now (if different than your previous answer to this question)?
2. How engaged do you feel right now on a scale from 1 (I am bored to tears! Save me!) to 5 (Shhh! Go away, I am busy!)?
This would allow us to formulate some baseline data to see where learning experiences fall in the overall spectrum of a learner's environment in terms of engagement. I am guessing that most of them fall into the bottom twentieth percentile, most of the time. If so, that can't be good. This kind of study could also be instrumental in identifying those exceptional learning experiences that are maxing out the scales. Think we might want to look at those particular experiences a little more closely?
Or imagine this variation: You are the instructor of a course. Each student has a little engagement meter, asking to rate engagement on a scale from one to five every five minutes during your class time. You videotape the class. You synchronize the video with the data. You chart engagement across time. Where you see peaks (hopefully) and valleys (inevitably) you jump to that part of the tape to see what was/wasn't going on. How much do you think you could improve your course after just one session of this? After three? Five? Do you have the courage to have the data reported to you in real-time while teaching the class? (Would that even be a good idea?)
If you know of anyone anywhere who is doing anything along these lines, please let me know. I am aware of classroom clickers, but not aware of anyone using them to measure engagement throughout class time to create an engagement graph. And I have never heard of anyone trying to establish an "engagement baseline" for learners that compares their learning experiences to the rest of the daily experiences in their life.
In closing, could you please rate your level of engagement with this blog post on a scale from one to five? ;)
Subscribe to:
Posts (Atom)
