Analytics Talk

Get Ready for Testing with Website Optimizer

April 4, 2007 by Justin Cutroni

Ok, I’ll bite. I’ll jump on the bandwagon and post about Google Website Optimizer (WO). It seems that everyone is writing about it today so why not one more post? :)

We’ve been using WO for over a year and love it. It’s easy to use, very inexpensive (actually free) and provides tremendous value.

But what I’d like to stress is not the product, but the PROCESS. All of us know that testing is just one phase in our beloved Web Analytics process. Theoretically we should all be testing right now or else we’re just finding problems and not improving visitor experiences. The great thing about Website Optimizer is that it gives us all a robust testing platform.

If you’re new to testing then I urge you to learn about the testing process before diving into Website Optimizer. I know it’s tempting to jump right in and start using the tool, but read about the process first.

Why Process Matters

Testing, more than almost any other part of the web analytics process, needs to be structured. Why? Because if you do not methodically define what you are testing and how you will measure success, you may not know if your test actually worked.

Here’s something I rant about all the time…

Many people believe that all tests are measured using one simple metric: conversion rate. I disagree. While conversion rate may be affected by your test, there are many micro-actions that can, and should, be tested.

For example, let’s say you want to test your add-to-cart button color. You’re testing the color of the button to invoke an action from the visitor. What is the action? To get the visitor to click the button and add the item to the cart. Yes, the action of adding the item to the cart influences the overall conversion rate by moving the visitor through the conversion process, but when you design your test you need to know which action you want to measure. The goal of this test may be to increase the number of visitors who add an item to their cart by 10%, from 50% to 55%.

Where to Start

So, what are some good testing resources? How about Avinash’s blog. He’s got a great post entitled Experimentation and Testing: A Primer. Start there and then head over to FutureNow. They’ve got some great books about how to actually test. You can find two great starter guides at their online store. If you’re already familiar with the testing process then check out the Website Optimizer help section and start reading.

You’ll start to see more posts on this blog about WO and testing. Some posts will focus on the process and some about the tool. Kind of like my approach to web analytics and Google Analytics.

If you’ve been working with WO please feel free to comment, I’d love to hear you’re reaction to the product.

Subscribe:

4 Comments »

Web Analytics: It’s About Process

December 20, 2006 by Justin Cutroni

Manoj Jasra asked me to write a small piece about the most common web analytics ‘pain’ that many of my clients feel. Thanks Manoj for the invitation! I’m honored that you asked. If you haven’t read the post you can find it here: Identifying & Solving Client Pains: Part 2.

Sure, many of my clients complain about data, specific KPIs and reporting. But I think those are common complaints that everyone hears. I think the biggest challenge my clients face is integrating web analytics into their standard business process. That’s right, I try to change the way my clients do business. What? You don’t know what this has to do with web analytics? Read on…

Web Analytics: The Process

We all know (or should know) that web analytics is a process. I’m not sure who first talked about the process (it was probably Jim Sterne or Eric Peterson) but it’s a pretty simple process that almost always involves the following steps:

1. Collect lots of data
2. Analyze the data (look for problems)
3. Create a hypothesis as to why a problem is occurring
4. Test solutions for your hypothesis (what’s the best way to fix the problem)
5. Implement change (the best solution discovered during testing)
6. Repeat
7. Make more money

The process must be fully implemented to work. For example, if you don’t test a hypothesis how will you know what solution is best for your problem? Each step is vital.

I believe that the above process must be integrated into an organization’s standard operating procedure. If it is not integrated then web analytics does not drive growth, it simply puts out fires. Here are some other thoughts as to why the process needs to be adopted:

• Accountability: By creating a process it is easy to identify what gets done and by whom.
• Scaling: Once the web analytics process is defined as standard procedure it can be staffed, managed and expanded.
• Credibility: If web analytics is simply ’something that Bob does as part of his job’ then it will never be recognized as a driver of change no matter how good Bob is.
• Protection for the company: What happens if Bob leaves the company? Who takes over and how do they come up to speed? If Bob follows a well defined process that is part of the standard operating procedure then it is easier to replace him. He becomes a cog in the proverbial machine. :)

How we do it

When we engage clients it’s for a specific amount of time (usually 6 months). During this time we have two goals. First, we want to deliver value using web analytics. We perform a number of small projects that focus on some type of optimization (SEM, SEO, email, etc.). Each of these projects is simply the execution of the web analytics process. We just change the focus of that process from month to month depending on what we’re trying to optimize.

Our second goal is to educate the client. This is where the organizational change comes in. We want the client to see that web analytics is a process that they can do. I believe that it is in the best interest of the client to adopt the process for the long term. We want all of our clients to use web analytics long after we’re gone.

What you can do

There is no set formula to help a company change it’s business processes. Every organization is different in the way they do business and their willingness to change (the later being the most difficult to manage).

In my opinion, the best thing that an analytics consultant can do is go into an organization and make an immediate difference. If you can show the client the value of web analytics you will gain credibility and hopefully a champion. Then start to broaden your effort.

After a few victories, talk to the decision makers about passing the process off to the client. Many companies fear change. Don’t be afraid to rock the boat and suggest that they take ownership of the process. If you’ve provided value using analytics then they will understand the benefit of the process and will listen. This is often a political issue. Be mindful of how the company does things. If they are a small group they may not have the capacity to take on another process. Again, it all depends on the company you are working with.

Remember, you are the web analytics expert. You know the process and have proven that (hopefully ;) ). The client knows their business and how they get things done. Now it’s time to integrate the two processes. I can’t tell you how to do that, that’s where the magic happens. I will say this, it doesn’t hurt to have some knowledge of business processes and optimization.

Other thoughts

I wanted to note that Eric Peterson mentioned this at the EMetrics summit in Washington, DC. Yes, the one back in October. Yes, I’ve been meaning to blog about this since October but have been ridiculously busy. I agree with Eric. I think this is the next challenge for our industry. Web analytics is an adolescent. For it to mature into an adult it must become a mainstream business process that drives growth.

Finding problems is so 2004. Discovering new business opportunities is where it’s at.

Subscribe:

8 Comments »