Creating a Hypothesis for Site Optimization

Creating a hypothesis should be one of the first things that you do when you start running A/B and multivariate tests on your Web site. Just because you have the keys to an optimization tool (even a free one like Google Web Site Optimizer), you should NOT be starting out saying, “hey, let’s see if changing this button from ‘Add To Cart’ to ‘Buy Now’ works better!” It’s vital to understand that you need to start with a hypothesis and then set clear goals before you start testing.

Setting a hypothesis is not a difficult thing to do, and it will help you stay clear on exactly what you are trying to accomplish in running a test. Here are a few examples of what might be appropriate hypotheses:

  • By changing the button on our product details page, we expect that we will be able to increase the rate at which visitors add products to their carts.
  • If we can decrease our shopping cart by one complete step, we can make it easier for customers to complete their purchase, thereby increasing conversion rate.
  • If we can provide more targeted information on our most popular landing pages, we can decrease bounce rates.
  • Maybe if we make it easier for visitors to use our internal search, visitors will more easily find products of interest, increasing conversion rate.
The common theme among all of these ideas/hypotheses is that none of them address, specifically, what will be done. This is the best way to start, because:
Creating and starting with a hypothesis, frees you from simply testing graphics and content, enabling you to test your business ideas and site effectiveness (i.e. conversion).

It is the hypothesis that you should be taking to the rest of your team when asking for the best user experience and design ideas to prove your hypothesis. You should not let a designer alone be the one that starts the process of site optimization.

Creating a hypothesis also makes it easier to measure the results of site optimization. If you start with just a design that is going to simply be “better than the last,” there’s no clear way to measure that. For example, if you were to change how you present your internal search results, is your success measure conversion rate, add to cart rate, product views or maybe average order value? There’s no real answer here, and starting a test without a hypothesis will result in a lot of debate over what success is when it comes time to evaluate the test.

Your hypothesis should make it clear what you are trying to improve, so that everyone can agree upon the success measure in advance of the test.

So if you start your testing and site optimization with an appropriate hypothesis, your goals and the eventual evaluation of your success should more easily fall into place.