User Focused A/B Testing For Quickly Improving Your Website

This new technique combines the benefits of user testing with A/B testing for significantly improving website conversion rates and sales.

There are two major problems with A/B testing for improving websites:

• It doesn’t tell you why your visitors preferred the winning version or give you any feedback. It only tells you which version won (if any).
• The majority of websites don’t even have enough traffic for A/B testing.

So I decided to create a user focused A/B testing technique to help you discover ‘why’ they prefer different page versions, not just ‘what’ they prefer. And it doesn’t require lots of traffic like traditional A/B testing does.

And more importantly, it gives you great feedback and insights for creating high-impact website improvements, meaning you will have a much better chance of increasing your website conversion rate and sales.

Create a variation of the page you want to improve

First you need to create the variation of the page that you want to improve. Ideally the improvements come from insights gathered from doing conversion research (web analytics, expert reviews, user testing, visitor recordings, surveys and polls), as this ensures better improvement results.

Begin the 5 steps of user focused A/B testing

Here are the elements of user focused A/B testing that you need for determining which page your users prefer — the original or improved page.

1: Do a ‘Preference Test’ at Usability Hub with 100 users.

UsabilityHub offer this great alternative to traditional A/B testing, and it allows you to gather user feedback on which version they prefer and most importantly, why they prefer it.

To create a ‘Preference Test’ simply take a screenshot of both pages and upload them to the tool, and then ask a question about which page version (or page element) they prefer, and then a follow-up question asking for the reasons why. This can be for a whole page, or for a specific thing like a headline.

To give you enough responses to analyze and gain insights from, I suggest testing 100 users. You will need 200 credits for this, which costs $200.

Exclusive deal:
I’ve partnered with UsabilityHub to offer this exclusive deal for 20 free credits.
Claim these free credits on this page using the RICH-PAGE-20 credit code.

It will be one of the best $200 you will ever spend for learning about your website user’s preferences!

To make sure the users match your target audience, you need to also set the demographics that you want them to be. You can target by age, gender, country, household income, education status, employment status, plus more.

You can also recruit users by sharing a link to your preference test, which you could do either via email to your customers or by showing it to your website visitors in a popup.

2: Record your visitors interacting with the new and exisiting page.
Next you need to launch both versions of your page so you can watch recordings of your visitors interacting with both of them to see if there any differences in how they use them.

To launch them both you will need to use Google Optimize do a split test, so 50% of users will see each version. It doesn’t matter that you won’t have enough traffic to get a A/B test result, you are just using it as a tool to launch both versions at the same time.

You can use Hotjar to do this visitor recording of the pages. To give you enough insights you will need to watch at least 100 recordings of the 2 pages. To find recordings of those pages you can just setup a filter in the tool to match the URL of them both. If you are using the free version and have a lot of traffic, you will need to limit the recordings to only work on those pages.

In the recordings look for variances of where they are moving their mouse, where they seem to be hovering over more, what they click on, and what they seem to ignore. Also look for any elements of each page they seem to have difficulties using, as the new page should try to improve on this.

3: Create a user test with 5 users for your target audience.
User testing helps you gain indepth qualitative feedback from visitors about your new page varation, and is delivered in the form of a screen recording of them using your website and audio of them answering your questions.

Use a tool like Userfeel to setup a user test with 5 users that match your target audience — it’s important to add prescreener questions for this, e.g. the users should be familar with what you are offering or have purchased previously.

When creating the user test you need to add a task for them to compare the two pages, and then ask them the following questions to get better insights:

• Which version is more engaging?
• What are your favorite elements on both page versions?
• Is there anything confusing on either of the pages?
• Is anything missing that you would like to see added to each version?
• Which version would be more likely to make your purchase?

Then add a few questions that relate to the elements that you are improving, for example, what do you think of the page filters on both versions?

This is the most important part of user focused A/B testing, and will give you the most detailed insights when comparing two pages. It is the most expensive part though — Userfeel charges $60 per user, so testing 5 users will cost $300. You can get this for 10% off by using this exclusive Userfeel discount link.

4: Send a survey to existing customers asking for preferences.
Getting feedback from visitors who have already purchased is essential, as they will have already experienced the current version of your website and will often have great feedback. You can use a tool like Hotjar to create this important survey.

To gain the best insights you will need to give them links to the current page and improved page, and ask the following questions in particular:

• Was there anything about the current page that nearly stopped you from purchasing?
• What are your first impressions of both pages?
• Which page version would have made it easier for you to purchase?
• Is there anything missing on either page that would have helped you?

You should also ask specific questions regarding what you improved.

You should ideally get at least 50 responses from customers to analyze. To get higher response rates you should offer a gift certificate or discount for anyone who completes the survey.

5: Setup a visitor survey to gain feedback on both page versions.
This short survey enables you to gather insights from people who are currently using your website — who are essential users to gain feedback from.

You can use Hotjar to create this survey and popup an invitation to start it (don’t show it too soon after visitors arrive though, as it will often annoy them, so I suggest after a minute or two).

In the survey you need to show them both page versions and ask them the following questions in particular:

• What are your first impressions of both pages?
• Which page version would increase the chances of you purchasing?
• Do you think there anything missing on either page?
• Is there anything confusing or hard to understand on either page?

You also need to ask questions relating to exactly what you have improved.

You should get at least 50 responses to analyze, and to increase your respose rate offer a chance to win a gift certificate or discount as an incentive.

Analyze the insights and declare a winning version

Next you need to spend a few hours reviewing the results to see which page version your users seem to prefer, and the reasons why.

To help you determine the winning page version, you should create a simple document that highlights the positives and negatives for each page.

This analysis will also give you great insights and ideas for improving the winning page even further. This is a good opportunity to make additional improvements, particularly ones that improve usability or clarity on the page.

Launch the winning version and monitor impact on sales

You should feel pretty confident now that you have discovered which page your users like better — hopefully the improved page you created. If users prefer that page to the current page, you need to launch it.

To ensure it does have a good impact you need to monitor the impact of it on your conversion rate and sales in Google Analytics over the next few weeks.

If you are lucky enough to have enough traffic, ideally you should run an A/B test on this winning page to determine that it has a positive impact on your conversion rate and sales.

If the new page doesn’t get very good feedback, you need to gather more conversion research (web analytics, expert reviews, user testing, visitor recordings, and surveys) to create better improvement ideas for it, and then do another round of user focused A/B testing.

Do this user focused A/B testing on all of your key pages

Don’t just stop after you have done this for one of your pages.

To get the best results from this, you should do this user focused A/B testing on each of your key pages to improve them too, particularly your top entry pages and conversion funnels like your checkout or signup flow.

Go ahead and try this for your next website improvement. I’m sure you will love the great results you quickly get on your conversion rate and sales.