Run A/B Tests To Improve Your Website
Intro into A/B testing
The idea behind A/B testing is straightforward: you don't have to argue with your team which page is better. Raw data from testing will clearly provide an answer.
But it's worth noticing that even though the idea of A/B testing is quite simple, it's very important to set the ground rules correctly, so that the results are accurate and comparable.
Check out how EA raised their sales by 43% with the help of A/B testing (Case Study)
Sign up for Optimizely
We recommend Optimizely for several reasons:
- This service is very easy to install and easy-to-use. You don't have to be a programmer to get started with Optimizely.
- We use Optimizely at Casual.pm, and we are really happy with the results so far.
When we started working on this how-to guide, Optimizely offered Free plan. That's precisely the reason why we have chosen them for our guide.
As of December 2016, they do not offer free plans anymore. Moreover there are no free A/B testing platforms out there that provide good user experience and solid analytics. That said, you can test Optimizely for Free for 1 month. Also, we've listed a couple of alternatives to Optimizely so that you have something to choose from:
Set your Optimizely account
Here is how it’s done:
- Go to the Optimizely Home page. Click the Settings tab, then the Implementation sub-tab.
- Copy the line of code above the Snippet Details, exactly as you see it. Don’t add http/https to the URL or make any other modifications.
Add snippet to your source code
Setting up the experiments
- Sign up Form
- Above the fold
Also, each experiment essentially consists of 3 main steps:
- Create a variation
- Set goals to evaluate the result: was the experiment successful or not
- Run the tests and analyze the results
Test 1: "Slogan"
Create a variation
- Navigate to the Home page and click "New Experiment" (this is where you will create a variation for your A/B test). Enter the URL of the website you’d like to test and name your experiment.Name your experiment:2. Using Visual editor create changes that you'd like to implement.3. In this variation we are going to substitute "Turn Your Projects into Awesome Visual Maps" with "Turn your boring task list into a visual project plan".
Set up URL targeting
This is a place where you specify the address of a page that you are going to test. Usually, startups have just one landing page, but if you have a more complex website, then you can run multiple experiments on different pages at the same time.
Set audience and traffic allocation
- Press "Options", then select "Traffic allocation". Take a look at the screenshot below.
- By default, all traffic is divided into two equal groups: 50% of users see the original version of the page, and other 50% see the modified version. High-traffic websites can set the limit for their traffic that participates in experiment.
- Sign up button clicks
- Button click (to measure the general engagement on our Landing Page)
Q&A and publish
Then click "Variation" on your Editor page and select "Preview". Besides, you should perform an in-depth QA review before pushing your experiment live to the world. Create a QA checklist that outlines all of the changes you’ve made. Your QA checklist should include:
- All goals that were added, as well as how each is triggered
- All functionality that’s been added (for example, a new button)
- Common use cases including expected user flows to and from the page
- Audiences that are eligible and ineligible for the test (e.g. certain location only)
- URLs of the tested pages
- Sample workflow to fire a goal (especially for custom events)
Run your experiment and analyze the results
Go to Home page and press "Results" to see whether your test proved the value of any improvements made to the original version of the site.
Our test results show that Engagement has slightly increased, but signups stayed the same.That's not much of achievement. Let's declare this experiment failed. However, it is important that we conducted an experiment and now operate with numbers instead of guesses.
Implement changes (if needed)
Test 2: "Sign up Form"
Signup forms are crucial in terms of visitor conversion into the potential client. Confusing forms will make you lose your customers. On the other hand, well designed forms may increase conversions and then sales.
Create a variation
Click one of the input fields and select Edit Element, then Edit Style. The "Edit Style" feature lets you easily make changes to the background color, border size, text fonts etc.
The Dimensions tab allows you to change the height and width of the input field. Once you have finished adjusting the sizes and input fields colors, click the text titles and repeat the process.
Click Options, then URL Targeting to set up the URL targeting conditions for this experiment. Since your experiment should only run on one page, simply paste your landing page URL.
Please note that we omit some steps (like Set Audience or Traffic Allocation), because they are basically the same in all experiments.
If you would like to collect more granular data, you can set up custom events that track when visitors begin typing in a specific input field. Custom events such as this one will allow you to evaluate whether a certain field is generating drop-off.
Test 3: "Above the fold"
It's important to put there clean and clear messages that won't distract potential customer from the following:
- Understand what are you trying to sell.
- What does the product do.
- How it helps and delivers value to the customer.
- How you can buy the service.
Create a variation
Version 1:Version 2:Second version generated 47% more sign ups! Why?
- Sign up page is clear, it doesn't confuse or distract, there is no images.
- The headline is straightforward and communicates the value better.
Set goals and run the experiment
If the experiment results in more sign ups on a landing page, then accept the new version of the "Above the fold" design.
Yet, if the amount of paying customers didn't go up, this would mean that there are problems with an onboarding procedure and you should focus your attention on customer experience after they sign up to your website.
In conclusion, we gathered a few articles that provide additional knowledge on A/B test:
- Get started on web optimization
- 10 Common Experiments and how to build them
- A/B Testing Case Studies
- Qualitative Questions That Lead to Great A/B Tests
- 15 of the Best Website Homepage Design Examples
- 12 Surprising A/B Test Results to Stop You Making Assumptions
- Useful Value Proposition Examples (and How to Create a Good One)