Jason Egan

Site Optimization, Web Analytics & Social Media

Archive for October, 2008

Creating a Hypothesis for Site Optimization

Creating a hypothesis should be one of the first things that you do when you start running A/B and multivariate tests on your Web site. Just because you have the keys to an optimization tool (even a free one like Google Web Site Optimizer), you should NOT be starting out saying, “hey, let’s see if changing this button from ‘Add To Cart’ to ‘Buy Now’ works better!” It’s vital to understand that you need to start with a hypothesis and then set clear goals before you start testing.

Setting a hypothesis is not a difficult thing to do, and it will help you stay clear on exactly what you are trying to accomplish in running a test. Here are a few examples of what might be appropriate hypotheses:

• By changing the button on our product details page, we expect that we will be able to increase the rate at which visitors add products to their carts.
• If we can decrease our shopping cart by one complete step, we can make it easier for customers to complete their purchase, thereby increasing conversion rate.
• If we can provide more targeted information on our most popular landing pages, we can decrease bounce rates.
• Maybe if we make it easier for visitors to use our internal search, visitors will more easily find products of interest, increasing conversion rate.

Creating and starting with a hypothesis, frees you from simply testing graphics and content, enabling you to test your business ideas and site effectiveness (i.e. conversion).

It is the hypothesis that you should be taking to the rest of your team when asking for the best user experience and design ideas to prove your hypothesis. You should not let a designer alone be the one that starts the process of site optimization.

Creating a hypothesis also makes it easier to measure the results of site optimization. If you start with just a design that is going to simply be “better than the last,” there’s no clear way to measure that. For example, if you were to change how you present your internal search results, is your success measure conversion rate, add to cart rate, product views or maybe average order value? There’s no real answer here, and starting a test without a hypothesis will result in a lot of debate over what success is when it comes time to evaluate the test.

Your hypothesis should make it clear what you are trying to improve, so that everyone can agree upon the success measure in advance of the test.

So if you start your testing and site optimization with an appropriate hypothesis, your goals and the eventual evaluation of your success should more easily fall into place.

---

New Google Analytics Features - First Impressions

So I have access to the new updates to Google Analytics now. My first impressions of the updates are very positive. The two big additions (aside from the API) are the custom reports and segmentation.

The custom reporting actually went a little beyond my expectations. The big thing here that Google has done is that you can create reports that are very deep (4 levels or so) and that you can correlate across different report dimensions easily. Now Google just needs to implement more variables and events to take this to the next level.

The segmentation works well too, but it’s a little clunky in it’s setup. Here, I like the buckets that you start with in Omniture’s Discover. The creation of a segment seems a little more intuitive when starting from the idea of a visitor, visit or page view.

I do think that with the additions of the custom reporting and segmentation that Google has become more of a serious option for larger businesses. 

---

Omniture Launches Developer Connection & Discover API

Okay, I’m not trying to make this an Omniture blog, but they keep releasing products/services and acquiring companies at startling rate the last few weeks. That being said, I received an e-mail from them this morning about launching a beta of something called the “Omniture Developer Connection.” At first glance, this appears to be just a repository for documentation on Omniture’s Web services. One BIG thing that I did notice, is that:

There is now a Discover API and accompanying documentation!

This is great to see, and I hope that Omniture gives some more attention to this new API. I would like to automate some Discover reporting, and an automated CSV file isn’t the most elegant way to accomplish it.

This site (that requires Omniture login credentials) also contains forums for developers as well as a library of code examples that can be contributed to by developers. Right now though, there are no posts in the forum (aside from the admin) and there’s only one code example.

Here’s the announcement e-mail from Omniture:

Announcing Omniture Developer Connection (BETA) 

The Omniture Developer Connection is here—a community Web site designed to help our customers build applications  that use their Omniture data. Found at http://developer.omniture.com, the Developer Connection allows our customers to:

• Use Omniture SiteCatalyst data across third-party applications, such as an intranet or a company-branded application
• Access SiteCatalyst reporting data to create calculated metrics, or format the data to meet specific internal needs
• Use the data collection API to facilitate the integration of SiteCatalyst with applications that cannot be easily tagged with JavaScript

In addition, Omniture Developer Connection contains:

• Documentation of Omniture’s application programming interfaces (APIs)
• Sample code showing reference  implementations to give developers a head-start in developing their own applications
• Discussion boards and blogs to provide peer-to-peer support among those building Omniture-driven applications

Please pass this along to the appropriate development team within your organization. For additional detail on the Developer Connection, a list of Frequently Asked Questions is provided below: 

1. When is the Developer Connection available?
Omniture will be releasing the Developer Connection in beta on October 17, 2008.

2. Who can access the Developer Connection?
The Developer Connection and Omniture APIs are accessible to Omniture customers with a SiteCatalyst login.

3. What are the Omniture APIs?
The Omniture Web Services API provides programmatic access to Omniture SiteCatalyst administration, data insertion, Omniture Data Warehouse and reporting functionality. The Web Services API is built using SOAP, which allows developers to use any SOAP development toolkit to start developing applications. The data insertion API is built on an XML-based schema, allowing developers to easily and quickly send data and begin testing integrations with the system.

4. How do I access the Developer Connection?
Customers can use their SiteCatalyst login information to access the Developer Connection at http://developer.omniture.com.
 
5. What is the best way to start using Developer Connection?
A Getting Started guide (for users with a SiteCatalyst login) is available to assist new users through the process of learning the prerequisites, enabling the Web services APIs, and testing and authenticating newly developed applications. The guide is available under the ‘Getting Started’ tab in the portal.

6. Does Developer Connection include API documentation for all products in the Omniture Online Business Optimization suite?
Currently, API documentation is available for SiteCatalyst, DataWarehouse, Discover reporting, and SearchCenter. Additional API’s will be added in the future. 

7. How should Beta participants provide feedback regarding the Developer Connection portal? 
We will be monitoring the community blogs and message boards and encourage customers to provide us feedback there. 

Sincerely,

Your Omniture Team

---

Omniture Acquires Mercado

I received an e-mail this morning that Omniture has acquired Mercado, one of the largest players in on-site search. The funny thing here is that Omniture has only recently started selling the rebranded VisualScience product (which was formerly a WebSideStory product) for site search. I am guessing that Omniture will take the same direction here as they have with their Discover product, making the VisualScience product a “lite” version compared to their newly acquired product from Mercado. Here’s the e-mail Omniture sent this morning making the announcement:

Update:

I’m hearing on Twitter that this was a bit of a fire sale (http://twitter.com/jbillingsley/statuses/959067564). It also looks like Mercado wasn’t doing so well and that the software-as-a-service business model wasn’t helping the issue (http://www.startupisrael.com/mercado-shutting-down). I know that Omniture uses the same basic business model, but they also have the advantage of a huge customer base.

---

Social Media in the Classroom

I was thinking today about the potential impact of social media on the traditional classroom environment. There were several thoughts that came to me about how social media is going to seriously change the education game for students that are utilizing it:

1. It can’t be stopped. Students are going to use Twitter, Facebook, MySpace, etc. whether educators tell them to or not.
2. In the past, the sharing of information in the classroom was considered cheating. Now, the Information Age is all about sharing and collaboration. Should the idea of sharing knowledge in the classroom between students change? It seems a little foolish to not utilize every resource you have in the information age.
3. Most young students are into mobile communication and social networking much more so than older educators. As devices like the iPhone eventually become the free phones we get with cell phone contracts, it will become increasingly important to integrate this technology into our education.
4. Many students may not have the finances or regular access to social media and tools such as the iPhone. Our education system will need to work on making these technologies and capabilities available, and not just too the elite few.
5. Students will be sharing information and looking for answers with students well outside the classroom and sphere of influence of educators. Who’s going to stop a student from Twittering with someone across the country to get help on a complex essay or homework assignment?

Do you think that our education system is ready to tackle the implementation of social media in the classroom? What other challenges can you see out there for schools as social media becomes more prevalent?

---

Creating an Action-Oriented Culture

Most experts in Web analytics dream of working for a business that makes data-driven decisions. Look at it as justification for our career choice. However, I think that this needs to be taken a step further:

We should be concerned with creating an action-oriented culture, not just a data-driven one.

Having access to advanced Web analytics tools, we can churn out meaningful analyses that management will most certainly use in making decisions. So, that means that we work for a data-driven company, which is a great thing indeed. The missing thing there is that the information isn’t nearly as useful if no action is ever taken. This can also be very defeating for people in Web analytics. No one wants to feel as though their hard work is never put to real, visible use.

So what can we do to help create an action-oriented culture when we don’t have direct control over resources and priorities?

Start A/B Testing

This is a slam-dunk, since action is one of the first steps here, and the payoff is pretty quick to see. Once you can get your company to buy into A/B testing (don’t worry about multivariate yet), more tests will follow as the return on investment becomes evident. Start easy with only A/B tests, where “B” is very different from A (to ensure meaningful results) and try out a tool like Google Website Optimizer (free) to help make ROI justification easier.

Focus on Change

Everyone delivers dashboards and standards reports. As I’m fond of pointing out, these eventually get ignored with all of the other noise that exists in inboxes. If you want people to stand up and take notice, and eventually act, then you’re going to need to get a little more manual and really dig into the “why” of your data and information. Like it or not, we Web analysts don’t know everything that marketing, Web design and the other teams on our site are doing. So, instead of just delivering dashboards on a regular basis, we need to start tracking significant changes in metrics, investigating with the necessary people, and finally delivering an intelligent analysis on the “why.” In addition, we need to make efforts to be more in the loop with all of our online efforts to understand and provide analysis on the efforts that are not making any significant changes on our site(s). After all, why waste effort on something that doesn’t even move the sales needle?

Understand Your Customer/Visitor

I just mentioned how important it is to understand the “why” in why things are changing or happening on your site. While traditional quantitative analytics is great, we must supplement our data with qualitative data. We must also do this at the visit level so that we can correlate the “why” to what visitors actually did (or did not do) on our site. If you can start to get survey input from your site visitors, then you can put some real, hard evidence that action needs to be taken and put it in front of the right people. These people might find it a little harder to not act on information from a person that has a proven purchase or visit history with your Web site. While there are a lot of survey solutions out there, you should at least start with something free like 4Q, which was created by Avinash Kaushik and iPerceptions. You may not be able to get the answers to the 4Q survey related to your transactional data (hey, it’s free!), but it’s a starting point in gaining qualitative knowledge about your visitors/customers.

Do you have any other ideas as to how data and information can spur action?

---

Why You Should Be Using Omniture Discover

This of course only really applies to current Omniture customers or those thinking about moving to Omniture as a new customer. I’ll also preface this further by saying that I am on Omniture’s Customer Advisory Board (CAB) for Discover (On-demand).

Anyway, there are several reasons that you should be using Discover and several reasons why you need it in addition to the normal SiteCatalyst offering.

Reasons why Discover is a good supplement to SiteCatalyst Include:

• SiteCatalyst’s power is in providing reports and delivereing those reports and dashboards on a regular basis. The power of Discover on the other hand is to enable you to perform true exploratory analyses.
• SiteCatalyst has limits in terms of how far you can drill down into data, and also limts in what you can drill down to. In Discover, you can drill down to anything, as deep as you like.
• Drilling down in SiteCatalyst only works from the moment it’s enabled, and is not retroactive. In Discover, everything can be drilled down to, and everything is retroactive. So if you think of something that you’d like to look at a few months ago, you can. Everything is retroactive from the moment you enable Discover though, not before.
• In SiteCatalyst, you can segment anything by using what are called ASI segments. However, ASI segments aren’t flexible enough that you can easily change the segments definitions themselves in the middle of an analysis. In Discover, you can change segments on the fly with a simple drag-and-drop.
• Omniture Data Warehouse can deliver segmented data to you, but can take 1 – 3 days to do so. In Omniture Discover, you will get your data in just a few seconds. This allows you to see the results of segmentation quickly so that you can know how to best craft a Data Warehouse request for large dumps of data.
• There is no additional implemenation for Discover. Once it is enabeled, all of your reports are ready to go.

So these are some of the reasons that Discover can supplement your use of SiteCatalyst.

So what are the things that you can do in Discover specifically that make it a great tool for deep analysis?

• Dashboards and regular reports are often and eventually ignored. Deep analyses of segmented data allows you to create presentations of informaiton that most others will not have access to.
• Quick creation of segments . Why look at just every visit on your site, when you can look at:
  • First time visits
  • Return visits
  • Visits that include certain pages on your site
  • Visits where your help content is viewed
  • Visits where no purchases are made
  • Visits where people fallout of your purchase funnel
  • Visits where people used internal site search
  • And many more…
• Omniture Discover is fast. When starting Discover (a Java application), it imports a data set. Once that’s imported, you need only wait a couple of seconds for anything to run. When using a Web-based application (as are most Web analytics solutions), you have to make a request of a server, the server gets the data that you need and then you are delivered the information in your browser. This can leave you waiting for a report until you are frustrated.
• If you are running A/B and multivariate tests, Discover will let you analyze test results in a deeper way that can make determining the winner more clear-cut.
• After SiteCatalyst has all of your reporting and dashboards built out, Discover is the tool where true Web analysts will work on a regular basis.

If you have any specific questions about Discover, let me know. Also, if you are already using Discover, do you have any particular segments or analyses that you like to look at on a regular basis? Any tips or tricks of your own you’d like to share? Leave a comment and share anything that you have!

---

The Economy and Web Analytics Jobs

A lot of people have lost their jobs over the last few months, and a lot more probably will. Most people in analytics know that there is more demand for skilled Web analysts than there is supply. So, would you say that experts in analytics have recession-proof jobs by default?

I keep seeing a lot of articles and content poping up on the Web about recession-proofing your job. Most of them say that you need to appear indespisible (perception is reality, right?) as well as be more of a jack-of-all-trades. Here’s a podcast from Harvard Business Review on this:

Harvard Business IdeaCast 110: How to Protect Your Job in a Recession

What can Web analytics people do to recession-proof themselves?

As I see it we can do several things at least:

1. Constantly (and I mean almost daily!) turn out ideas to improve your company’s online presence
2. Present to management often (to increase your visibility and the perception of being indespensible)
3. Speak at conferences (analytics or otherwise) and become better know in your company’s own industry (unless the bad economy has resulted in your travel budget being shot!)

One other thing that you can do to recession-proof your job is to keep some information close to the vest. That is, don’t run out and tell everyone in your company how to do everything that you do. If you document exactly how to do all of the technical aspects of your job, this might also hurt you in recession-proofing your job. I can’t say that I really agree with everything here about not sharing things about your job skills, but I’m just throwing it out there as something that we all know is done in reality.

Do you have any further ideas as to how people in Web analytics can help themselves in recession-proofing their jobs in this economy?

---

Social Networking in the Business

Several companies have released as of lately social networking applications targeted at business users as opposed to the general public. Some of these companies/applications include:

Since I have become a regular (maybe addict is a better term) user of Twitter, I find that I now rarely use e-mail, and I haven’t even logged into an instant messenger program in weeks. The idea of microblogging seems to just work better for collaboration and communication than does email or IM. On Twitter, I made a comment about Present.ly being more like e-mail 2.0 than just another Twitter clone:

http://twitter.com/jasonegan/statuses/926264496

Shortly after making this comment, Present.ly picked up on my tweet and promptly answered back:

http://twitter.com/presently/statuses/926317568

Microblogging will become as ubiquitous as email and IM.

However, while I do think that microblogging will become mainstream in the future, I think that these companies may be a little ahead of their time. Of course, we all know that Internet ages like cat years, and the time will be sooner that we may think. It may also take some time for companies to be willing to pay for something called social networking.

Unfortunately, corporate America in many cases doesn’t want us doing anything “social” while on the job.

Afterall, they’re not paying us to be social, right? Most companeis will have to see communications solutions like these in practice at large companies, or see them used at a conference in order to drink the social kool-aid.

What are your thoughts on the possibilities social networking gaining traction in the workplace? What hurdels do you see?

---

Powered by Wordpress | WP Premium theme by WP Remix
Copyright 2007. Jason Egan. All rights reserved

• Home
• About Jason
• Google Analytics Resources
• Resume