The Chocolate and Peanut Butter of Learning Analytics

On October 23rd, 2009, Adobe Systems, Inc. competed its acquisition of Omniture, Inc. An Adobe press release describes the benefits of this merger as follows:

The combination of the two companies will increase the value Adobe delivers to customers. For designers, developers, and online marketers, an integrated workflow — with optimization capabilities embedded in the creation tools — will streamline the creation and delivery of relevant content and applications. This optimization will enable advertisers and advertising agencies, publishers, and e-tailers to realize greater ROI from their digital media investments and improve their end users' experiences.

Beneficiaries: advertisers, advertising agencies, publishers, etailers
Benefit: greater ROI, better end user experience

What this press release does not say is that on that fateful day the chocolate of Omniture's web analytics tools dropped smack dab into the middle of the peanut butter of Adobe' s eLearning suite. Or it could have/should have/will if someone stops trying to flip up the most enticing Flash banner ad possible for a minute and thinks about the eLearning world. If the red and black eLearning folks will wander over to their new 1.8 billion dollar, lime green roommates and make a modest proposal, we could have a match as classic and enticing as the Reece's peanut butter cup.

Does Adobe have any idea that they are sitting on the biggest revolution in eLearning since the browser? Do they realize that they now have at their finger tips all of the tools necessary to dominate the eLearning space with the hottest, most completely integrated, most elegantly implemented learning analytics suite on the market? Who else has the power to build learning analytics straight into the most popular tools for elearning design and development? Who else has the statistical and number crunching guns to process and display massive amounts of learner data in slick, easy to use dashboards? Could there be a more obvious fit? Does anyone there realize that they could be the engine that powers a massive emerging industry?

Probably not. Why would the people focused on ROI for advertisers start scribbling on the back of napkins with people who are focused on ROI for eLearning ? Nothing against Adobe; big corporations just don't innovate this way very often. If they did, the press release might read as follows:

The combination of the two companies will increase the value Adobe delivers to learners everywhere. For instructional designers, eLearning developers, and online colleges and universities, an integrated workflow — with learning optimization and tracking capabilities embedded in the creation tools — will streamline the creation and delivery of customized learning content and experiences. This optimization will enable teachers, trainers, instructional designers, and training organizations as well as online educators to realize greater ROI from their digital training and teaching investments and improve their end users' experiences and, ultimately, the overall appeal, effectiveness and efficiency of their learning.

It would be a crying shame if this didn't happen. Imagine a world without Reece's peanut butter cups. It would be that bad.

Web Analytics in Education

A few months ago, I saw a presentation by Clint Rogers on web analytics at BYU. Since then, I have been thinking a lot about the possibilities of web analytics for education. I even told my friends that I am pretty sure that whoever figures out how to do it right first is going to start an industry. I was pleased to see that Clint was teaching a seminar on that very topic this fall at BYU so I am sitting in on it even though I have almost no spare time right now. As a result, several upcoming posts will be related to this topic.

So what could we learn from web analytics that we don't already know and what could we do with that knowledge?

My brainstorm:

• We could know exactly who looked at what and for how long. We could know which of the 10 things we thought they absolutely had to read they actually did read (or at least left open on their browser) and for how long and then correlate that to their scores to see if they really did need to read those ten things or not.
  
• We could find out if the $5000 simulation we built gets more actual student face time than the $500 game.
  
• We could provide approach A 50% of the time and approach B 50% of the time and correlate to outcomes to see if one has better results.
  
• We could identify learners who are not logging in, or clicking randomly, or only doing the quizzes and intervene by notifying them automatically (but as if we are human) that we have noticed this pattern and we are concerned (a human would read the reply, of course).
  
• We could possbily identify profiles of people who are cheating.
  
• We could find out if online students really do cram the entire course in to the last three weeks of the semester and still get an A- on the final and reflect on how we feel about that.
• We could discover that you only need to skim this particular course to get a B-.
• We could discover that if you only read the intro and the summaries of each lesson you get a passing C.
• We could discover that those who do all the optional quizzes and pace themselves so that they complete three lessons a week get an A and then tell new students at the beginning of the course of this pattern for success in this particular course to help them invest in good study practices. And, if they fall off the wagon, we could remind them that their current, not so hot learning patterns correlate with a D for 90% of the students last semester that fell into this pattern and didn't change by October 1st. In fact, profiling the behavior of high performing students or of those who get off to a rough start and recover or of those who spend the least amount of time in the course but get the highest grades or, or, or..., I think, is one of the most interesting areas that could be investigated and could lead to a lot of good advice for others taking the course and entire course redesigns to make them more lean and mean and precisely helpful. Especially if we can profile the students entry characteristics and then correlate them to success patters for those specific characteristics.

  "Dear student, According to the survey and your past grades, you are very similar to 86 students who took this course in the last 2 years. These students also 'enjoyed working on their own' but 'felt that they learned slower than most' and had similar grades to you on the pre-requisite courses. Students with this profile were most successful in this course when they followed these study habits: yada yada However, most of these students were more inclined to follow these less effective patterns: yida yida. We have sophisticated tools that can produce a weekly report showing how close your study habits are to those of students with your profile who were sucessful in the past and warning if you fall into the less effective learning patterns common to students with your profile. Would you like us to send this report to you?"

None of this feels like TLC for the student, but I believe that the hard numbers and statistical patterns can be presented in a very human, non-threatening, helpful way that really will help students feel like the course designers/instructors know them and are there to help them and have this almost magical insight into how they can improve their performance in the course. Maybe not. But it is very much worth a try.