I want to conduct this study:
Twenty college students (or learners of your choice) are given a means and a prompt (and a reward) to answer the following two questions every five minutes for all of their waking hours over the course of a day/week:
1. What are you doing right now (if different than your previous answer to this question)?
2. How engaged do you feel right now on a scale from 1 (I am bored to tears! Save me!) to 5 (Shhh! Go away, I am busy!)?
This would allow us to formulate some baseline data to see where learning experiences fall in the overall spectrum of a learner's environment in terms of engagement. I am guessing that most of them fall into the bottom twentieth percentile, most of the time. If so, that can't be good. This kind of study could also be instrumental in identifying those exceptional learning experiences that are maxing out the scales. Think we might want to look at those particular experiences a little more closely?
Or imagine this variation: You are the instructor of a course. Each student has a little engagement meter, asking to rate engagement on a scale from one to five every five minutes during your class time. You videotape the class. You synchronize the video with the data. You chart engagement across time. Where you see peaks (hopefully) and valleys (inevitably) you jump to that part of the tape to see what was/wasn't going on. How much do you think you could improve your course after just one session of this? After three? Five? Do you have the courage to have the data reported to you in real-time while teaching the class? (Would that even be a good idea?)
If you know of anyone anywhere who is doing anything along these lines, please let me know. I am aware of classroom clickers, but not aware of anyone using them to measure engagement throughout class time to create an engagement graph. And I have never heard of anyone trying to establish an "engagement baseline" for learners that compares their learning experiences to the rest of the daily experiences in their life.
In closing, could you please rate your level of engagement with this blog post on a scale from one to five? ;)
6 comments:
Joseph,
Millions of people are already doing this; it's called Twitter. They may not be capturing exactly the data you're asking for, but I swear there are some people who tweet every 5 minutes. Are you a Twitter user?
I like your gumption to try something like a real time measure of engagement. However, I have some concerns that asking people to think about their engagement so frequently will become a disruption to the kind of engagement that is absorbing. Can we really gauge our engagement in a trustworthy way constantly in the moment? Or is engagement something that develops in other temporal dimensions other than in-the-moment?
Joseph,
Fascinating idea! I agree that asking about their engagement will be somewhat disruptive to their actual engagement, but also believe you would get some invaluable information that would improve instruction. Interestingly, it would also make the users more self aware of their own engagement levels, possibly causing them to be even more dis-satisfied with boring presentations on the one hand, and more interesting presenters on the other. If you ever do it let us know.
Thanks for the comments. I am not a Twitter user but have investigated the technology a little bit. That might be an interesting avenue for collecting data. Thanks for the idea, David.
I agree with Pat that engagement is more than just moment-by-moment absorption. And engagement for engagement's sake is not always good. Studies in the learning sciences have shown that the kind of engagement tactics that are essentially sugar-coating don't promote learning (and can even denigrate it).
I agree with both Pat and Kent that reporting engagement so often will change the mindset of the learner and could disrupt their learning. But I think I am with Kent in thinking that it is still worth it for what we could learn from it. At least the study should be done once, and maybe it could be done once a semester and not be too distracting. I mean, in the classes I am in, most of the students are multi-tasking anyway...emailing, texting, even updating their Facebook profile. Sometimes I wonder if "the kids these days" even know what it means to not be distracted.
Hey Joseph, in response to your question on http://veletsianos.wordpress.com/2008/03/28/how-do-we-design-for-learning-engagement/ and related to this post - i think the value of a qualitative kind of an approach is that it may reveal prominent directions and may open further doors of exploration. If you are set on the quantitative approach, I would echo Pat's comments above, but would also add a qualitative approach. Let me limit your study to 45-minute lesson (or a 3-hour class) for a second: Taking a case study approach you can consider the class to be your case. You could then conduct a focus group on the experience of engagement during the course. You could obviously vary things in the class - you could for instance have 5 different kinds of tasks, varying in duration, pedagogy,and other variables that you think may influence learner engagement. By talking to the students afterwards you may be able to tease out further items of interests, for example, by asking students "Why do you think you were more involved in X rather than Y?" These are just a few thoughts.... I'm sure they can be improved :)
George,
Thanks for the thoughts. I really appreciate you taking the time to write. The case study idea is an interesting one and would, of course, lead to a much richer data set. We would have to find a pretty cooperative set of students to provide regular and sometimes detailed feedback, but I think that could be feasible. I will bring up this idea when we have our initial meeting about the study. Thanks again.
Post a Comment