Posts Tagged ‘A/B testing’

Testing Your Site (A/B testing)

October 30th, 2009

from

Find out what works best by using A/B testing on your website.
By Melissa Campanelli   |   Entrepreneur MagazineApril 2006

Not long ago, A/B testing was solely available to large companies–it was time-consuming, expensive and required much IT help. But now, many companies offer hosted testing and optimization services affordable to smaller businesses.

That’s good news for e-tailers seeking to improve their sites. A/B testing (aka A/B split testing) allows you to test two versions of your website. You can measure how customers respond to both versions and then optimize the site based on that information. Kefta, Offermatica and Optimost are three leading companies that offer A/B testing; services run about $10,000 per month.

RingCentral Inc. in San Mateo, California, is one growing e-tailer that’s had success with A/B testing. The multimillion-dollar telecommunications firm targets small businesses and sells its services exclusively online. Last summer, the company worked with Offermatica to run some A/B tests with the goal of boosting the percentage of visitors signing up for its services.

RingCentral tested various elements of its site, including the home page. Two versions were prepared: one with a laundry list of features–more graphics, links to a user guide and FAQs–and one with a detailed description of the benefits of working with RingCentral. RingCentral learned that the page with more links to additional information led more people to the sign-up page.

After changing the site accordingly, the 35-employee company saw a 46 percent increase in overall sales by year-end. “Thousands of people look at our site each day, and we have many ways we can present information to them,” says Vlad Shmunis, RingCentral’s 45-year-old founder. “A/B testing lets the public vote–basically with their wallets.”

To be effective, A/B testing should be ongoing. Some additional tips from Offermatica CEO Matthew Roche:

1. Run tests on the registration or shopping cart page. Says Roche, “Improving these areas generally provides the greatest impact on the bottom line.”

2. Focus on big elements. Instead of testing text color or font size, says Roche, focus on areas such as product, pricing, primary copy, images, offers and calls to action.

3. Test heavily trafficked campaigns in two weeks or less. Says Roche, “This creates better answers fast.”

Originally published in the April 2006 issue of Entrepreneur Magazine

A/B testing (aka A/B split testing)

October 30th, 2009

from: http://en.wikipedia.org/wiki/A/B_testing

A/B testing, split testing, or bucket testing is a method of marketing testing by which a baseline control sample is compared to a variety of single-variable test samples in order to improve response rates. A classic direct mail tactic, this method has been recently adopted within the interactive space to test tactics such as banner ads, emails and landing pages.

Significant improvements can be seen through testing elements like copy text, layouts, images and colors. However, not all elements produce the same improvements, and by looking at the results from different tests, it is possible to identify those elements that consistently tend to produce the greatest improvements.

Employers of this A/B testing method will distribute multiple samples of a test, including the control, to see which single variable is most effective in increasing a response rate or other desired outcome. The test, in order to be effective, must reach an audience of a sufficient size that there is a reasonable chance of detecting a meaningful difference between the control and other tactics: see Statistical power.

This method is different than multivariate testing which applies statistical modeling which allows a tester to try multiple variables within the samples distributed.

Companies Well-Known For Use of A/B Testing for Website Optimization

Many companies use the designed experiment approach to making marketing decisions. It is getting more of a common practice as the tools and expertise grows in this area. One study suggests up to two-thirds of web companies are dabbling in A/B testing.

While it is widely used behind the scenes to maximize profits, the practice occasionally makes it into the spotlight.

  • Amazon.com. Practically pioneered its use within the web ecommerce space. Also stirred controversy by testing into optimal price-points].
  • Google. One of their top designers left and spoke out against excessive use of the practice.
  • Intuit
  • Microsoft
  • Ebay