Saturday, December 6, 2014

Usability A/B testing

If you work in usability, you should be aware of A/B testing: presenting two versions of a product to a test participant, and asking which they prefer.

A/B testing is a popular and easy way to improve a system through iterative testing. By its nature, A/B testing requires some iteration to get it right: Think about when you visit the eye doctor, and you look through the sets of lenses: "Which looks better, number 1 or number 2? Now, number 2 or number 3?" And so on.

This style of testing can be extended to websites and web applications, by releasing one version of a site to a subset of visitors, and another version of the same site to a different subset of visitors. For example, a website might expose certain users to a different navigation sidebar, and compare how well those users can navigate the website versus those users who see the old navigation sidebar. With enough web users, the web designers can quickly determine (through traffic analysis) which navigation sidebar is more effective.

A recent article in Fast Company demonstrates another company using A/B testing, in an interesting way: the Adore Me lingerie company. From the article:
For each bra, Adore Me shoots multiple versions of images to run on its website. The distinctions between the pictures might include different models wearing the same set in the exact same position, or the same model in the same set in a different position, for example. Then, like any good tech company, it tests the options to find out which one sells better.

Through its research, Adore Me has found that the right model matters even more than price. If customers see a lacy pushup on a model they like, they'll buy it. Put the same thing on a model they don't, and even a $10 price cut won't compel them. Pose matters as well: the same product shot on the same model in a different posture can nudge sales a few percentage points in either direction. Another test found that a popular model can sell a more expensive version of the same garment.

The tinkering—which seems incremental—adds up, and has helped the company maximize sales. In four years, Adore Me has matched the sales of competitors like La Perla, bringing in $5.6 million in revenue, according to Inc.

Aside from model, A/B testing also revealed the model's pose can influence sales, too: "Hand on hip, a popular pose among Instagrammers trying to make their arms look skinny, doesn't resonate nearly as well as a hand on the head, for example. (That slight change can double sales, according to internal research.)"

This style of usability testing requires coordination between several groups: most importantly, the website must be geared to support traffic analysis in a way to help determine the best sellers: "For every thousand people that come on the site, 500 will see picture A, another 500 will see picture B and over time, one will sell better than the other."

As you plan your next usability tests, consider how A/B testing might be applied to examine how users interact with your system. If you crunch the numbers, you can discover interesting and useful results that will improve the next iteration of your product.
image: Adore Me

No comments:

Post a Comment