What Execs Want…

Best line I’ve heard in a while, from Ben Robertson (senior consultant in the High-Tech division at Omniture…yeah, I think I’m older than he is 🙂 about how to deal with the common resistance to tracking new metrics in web analytics. You know, metrics besides “unique visitors” and “pageviews” which most executives (and most analysts for that matter) simply call “traffic.”

Company IT Manager: We can’t! That’s what our executives want to see!
Omniture Consultant: Not it’s not! That’s just what you’ve been giving them.

His advice: go ahead and give them what they’re asking for, then give them what they should be asking for.
Hmmm…

More Analytics Musings…

This has really got me thinking. Take, for example the opening page of BYU’s Math 110 independent study course, entitled Special Instructions. Time-on-page for these users seemed fairly cleanly [though not evenly] split between those who simply skipped or made a quickly-abandoned attempt at scanning the page [spending 30 seconds or less] and those who put forth the rather ponderous 4 minute [an eternity online] effort to read the entire page. If 75% of our users really are “scanners” [then again, we only think we know this] then perhaps they [and, for that matter, we] would get more out of our site if we designed the content to be scanned.

Continue reading

Does Data Belong in the Driver’s Seat?

I’m struggling with this one. Our web analytics class is looking at a couple months’ worth of data from BYU’s most popular independent study course, Math 110 [not sure what the definition of “popular” is in this case, by the way] and making some recommendations, both about their tracking suite and about the course itself. Clint explained, and I understand, that analytics is not meant for examining a handful of people—it’s for looking at trends, types, aggregates. But, for me, that aggregation leads to serious questions.

Continue reading

You might be an early adopter if…

As if we needed more evidence that the education field has not, shall we say, fully capitalized on the potential of web analytics to improve instructional design and learning, I offer the following observations, in no particular order, from my recent search permutations on the subject:

  • #1 most common result: companies offering education and training on the subject of web analytics.
  • Second; pundits, bloggers and reporters bemoaning the dearth of educated web analyzers.
  • Next, a paper exploring what looks like an independently-designed user behavior tracking program for use in distance education evaluation. [If it turns out to be interesting, more later…]
  • Also, a big-news announcement that the world’s largest education company has chosen Omniture to help them optimize….their international marketing efforts.
  • Finally, Joseph’s post. Really insightful brainstorm on the subject and #3 in the google results for “web analytics in education”

This is not to say that there’s nothing out there…just interesting to see where the focus is right now–and just how much room there is for innovation in this area.

My Metrics Wishlist

This post is long overdue. Sorry, Clint.

The following are a few examples [untainted by any sort of experience with the subject matter, and irrespective of what is currently regarded as possible] of metrics I would find useful for gauging and improving the effectiveness of an instructional website:

Order and Timing: My primary frustration with the rudimentary analytics I have encountered so far has been a lack of correlation between behaviors. It seems that every minute piece of behavior is excised from the holistic experience and examined under a microscope independent of any of the surrounding behaviors. Not only that, but significant decisions are reached based on these dissected data. I want to be able to track navigation patterns with time spent per page and so forth.

Complex Behavior Visualization: The gist of this one is the creation of a three-dimensional map of the site allowing the designer to follow the path of a user [or an aggregate of users] through the site; streams of color that indicate the speed of transitions between pages, the number of times a page was hit, search terms, results and which one was chosen; size differences indicative of relative popularity/traffic; etc.

Comparative Paths: I’d love to be able to filter and compare the behavior of different target populations. Even just knowing how users referred from Google interact with the material differently than those who type in the address directly could be insightful. In an ideal world, I would be able to sort users according to target profiles [could be hardware, could be navigation style, could be socio-economic status] and examine the patterns of their interactions with the site; which links were clicked in what order, how much time was spent on which pages, what “conversion” goals were reached by each group, where and when did they bail?

Diverting the Stream of Consciousness: Having experienced first-hand the sometimes dramatically counter-intuitive insights provided by eye-tracking and think-aloud usability testing protocols, I would love to allow users the opportunity to opt into a remote usability lab environment, where the machine would capture eye motion and any verbal feedback the user cared to offer, even offering occasional prompts and feed it into a real-time database. Emerging themes would be automatically flagged and the designer could set up filters according to demographics or any other standard metric; entry point, referring site, time on page etc. to uncover patterns and monitor the effects of changes.

They’re all probably crazy–but you asked.