The Impact of A/B Testing

If you’ve ever wanted to be a social psychologist or an experimental scientist, then A/B testing is going to be even more exciting than it already is. Fair warning, A/B testing is my absolute favorite part of website design. It’s a hoot and a half!

What is A/B Testing? It’s pretty simple. It’s playing with the design of your webpage/app/whatever you’re trying to optimize. Basically, you tinker with your original design, come up with something new, and then expose half your website/app/ traffic to the original (control) and the other half to the reworked design (variation). After that, you’ll use an analytics program, like Google Analytics, to give you the data and the results of your little social experiment. The odds are that you’ll see a significant difference in the interactions, sales, or time spent on your page with one of the designs.


Use Google Analytics, don’t be ScumBag Steve! 

*It’s important to note that you should only change one feature at a time on your variation design! That way you’ll know exactly what is performing better/worse. 

Depending on what type of company you are, your A/B testing is going to look a little different. For example, an eCommerce company is going to want to tinker with the checkout/cart pages. A media company is going to want to play around with their recommended content. So it’s all based off of what your ultimate goal is — what your macro goal really is.

A/B Testing is incredibly valuable. It’s not just for regular Joe’s like you and I. Groups like Obama’s 2012 Campaign used it,  and WWF uses it too. Let’s talk about Obama.

The Obama Campaign worked their asses off to make sure Mr. Obama secured his position as POTUS. Over 20 months, The Obama Campaign website launched about 500 A/B Testing pages, and proved us once again that determination really does pay off. The campaign used time-tested approaches to see if their trials would result in an increase in donations and signed up (Spoiler Alert: It absolutely did). They removed the dollar $ign from the donation amount page and saw no statistically significant difference between the two pages, so they moved on. Every A/B Test isn’t going to be perfect, even when Obama is putting his magic into it.
The next test they used was placing a lovely picture of POTUS and FLOTUS as a variation on a web page and witness a spike in email sign ups!
Perhaps the most useful piece to take from the Obama Campaign website was the donation page. In their control, they had all the information forms (email, address, credit card info) on one page. The variation was to make it step-by-step… Which resulted in a 5% increase in their conversion rate! A simple fix, a huge impact.

Perhaps the most surprising example of A/B testing is EA Sports, who I’ve seen highlighted several times on various websites. The first place I read about EA Sports A/B testing was on HubSpot. In this case, the removal of a banner that displayed a promotional banner was actually working against their goal (sales). Removal of the banner on the left (Pre-Order And Get $20 Off) resulted in a whopping 43.4% more purchases! That is significant, tangible, and was all thanks to the fun of A/B Testing.


While there are several more components that must be accounted for before you launch yourself into an A/B Testing experiment, it’s definitely vital to optimizing your webpage. For those interested in learning more, I would suggest you read Optimizely’s A/B Testing article, which goes into much more detail.

That’s the fun of A/B Testing, and now you can be a website owner and a burgeoning social scientist.
I’m going to watch the last season of Parks and Recreation on Netflix and fall asleep at a decent hour for once! Being a student and an employee is…exhausting. Can I A/B test my life and see if I do better in school without a job? (Actually, I heard that having a part-time job has a correlation with higher GPAs..weirdly enough)


Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s