The history of web analytics has read a bit like the quest for the Holy Grail. We've gone through a list of candidates searching for the one true metric: Hits, Page Views, Visitors, Unique Visitors... stopping at each one to admire its purity and virtue while denouncing the heresy of whatever metric it replaced (usually, one whose purity and virtue we were just praising the week before).

While drinking from the wrong Grail in analytics won't melt your face like the bad guy in Indiana Jones 3, you may wish for some face-melting when you have to tell your boss how much money your bad conclusions just cost the company. This post will help you get control of your unhealthy obsession with Conversion Rate and avoid the most costly traps.

Conversion Rate Crash Course

Let's start with some basics, both for the newcomers and because the industry doesn't always agree on how to define terms:

There are many variations on conversion rate, and "Action" can mean just about anything – a click, a form submission, an RSS subscription, an actual sale – but let's keep it simple for now. So, let's say that for February your site received 10,000 visitors, and 450 of them took action:

Pretty simple, right? Don't get me wrong – conversion rate is powerful, and it captures an important bottom-line measurement. Problem is, it's just one number (well, ultimately, two numbers). So, what's missing? To answer that question, I'd like you to consider three scenarios...

Scenario 1 – Sacrificing Traffic

This is a situation that comes up frequently in PPC management – cutting traffic to raise your conversion rate. Here are a few examples to illustrate the point:

All three of these cases have 5% CR, so they're all the same, right? Of course not - all else being equal, anyone in their right mind would pick (C). Where people get into trouble is when they over-optimize for CR at the expense of traffic.

For example, let's say you have a classic PPC scenario: (A) a campaign targeting branded keywords with low traffic and high CR, and (B) a campaign targeting product keywords with high traffic and low CR. Your client starts complaining about low CR, so what do you do? You cut spending in Campaign (B). CR goes up, but the unfortunate side effect is that traffic goes down and overall Actions (read that "sales") go down with it.

SOLUTION:
Pay attention to both conversion rate and overall leads or visitors. Once you collapse down to CR, you've lost the top and bottom numbers and are left with just a ratio. If you're a PPC manager, set an acceptable Cost-Per-Action (CPA). Traffic within your CPA limit may be worth going after, even if CR isn't ideal – traffic that costs more than your acceptable CPA may have to be sacrificed. Don't just start chopping visitors to see CR go up.

Scenario 2 – Dropping Prices

Want the secret to increasing conversion? Cut your prices in half. What's that? You say you'll make a lot less money that way? Yes, you probably will. Of course, you'd never do anything that radical, but many people create sales, price pressures, and information architectures that drive people to the cheapest product. This can boost CR but cost you money.

Let's look at an example – say you get 1,000 visitors per day, and experiment with pushing a cheaper product ($29) over a more expensive product ($99) to boost CR:

Looking at the CR, it's great news: you doubled conversion. Unfortunately, your revenue also dropped 40%. There may be times when you're willing to make this trade-off to draw in new customers, but make sure you have all of the information you need to make that business decision.

SOLUTION:
If you make a change that could drive visitors to lower-priced items, make sure you track not only CR but also changes in the average purchase amount. If you're running an A/B testing scenario, consider tracking the mean or median purchase for both groups (use the median if your products span a wide price-range).

Scenario 3 – Losing Loyalty

An aggressive push to drive short-term conversion, including the pricing scenario above, could also lead to a drop in long-term revenue and customer loyalty.  If you offer a sweetheart deal that pulls in new customers, it's possible that they'll take advantage of that deal and disappear forever. Today's Conversion Rate gain, if it's driven by bargain hunters or impulse buyers, could be next month's Conversion disaster.

That's not to say that sales and short-term incentives are never a good idea. Driving traffic in the front door is essential to building long-term relationships. The core point is that, whenever you take an action that may change the quality of your customers (and not just the quantity), you need to look at the big picture.

SOLUTION:
These metrics are a bit beyond the scope of this post, but there are a number of Key Performance Indicators built around repeat buying and the lifetime value of a customer.  Whenever you pursue a short-term strategy, don't just measure CR, measure whether those new buyers are one-hit wonders or have real staying power.

It's Still Pretty Good

I don't want to sound like I'm bashing Conversion Rate. I use it every day and have driven real, bottom-line improvements for clients based on CR metrics. We just have to remember to never get so enamored with one metric that we neglect the big picture. Every web metric that has ever existed or ever will exist is missing some critical piece of information for some set of situations and has the potential to lead us astray. Think about your objectives, think about the possible outcomes, and most of all, think about all of the analytics tools you need to see that big picture.