When Multivariate is better than A/B Testing

When Multivariate is better than A/B Testing


Testing two or more variables on a site is possible through A/B or multivariate testing. The former is considered a more simplistic approach whereby traffic is split evenly to test two different versions of a web page. It’s also possible to test three or even four versions, but that would be known as an A/B/C/D test. With that being said, it’s a great tool for testing the design of a landing page, discovering which overall layout and text-to-image ratios work best.

Although, it’s the most utilised testing practice, many webmasters are now looking to the multivariate method for more insightful results.

What is multivariate?

Unlike the traditional A/B test, multivariate takes a subtler and specific approach. Rather than testing different versions of a web page, this is a test of the individual elements within a given page. In that regard, it tests multiple variables to uncover the best combination of elements. In other words, how do these work together to deliver optimum results in terms of conversions and user engagement.

Due to the nature of multivariate testing, webmasters have much greater control over the variables. As an example, you may wish to test three different designs of a sign-up form, along with two different headlines and three choices of engaging images. Traffic would then be funnelled to all possible combinations of these variables.

At the end of the test, all the information can be analysed to determine which combination was the most effective. It can highlight those minor details which actually have a major impact. A certain font on a headline could be frustrating users while an oversized sign-up box could deter visitors from joining your mailing list. When you consider the implications of design, multivariate testing seems like an attractive marketing tool.

When should it be used?

Although multivariate testing is clearly an important strategy, it can have its limitations. To carry out the test, you’ll likely be splitting traffic between ten and twenty combinations or more. Even well-established websites may find it difficult to complete the test in a reasonable timeframe given the amount of traffic needed.

Due to this constraint it can be more costly and time consuming than A/B testing. In some cases, a series of well-executed A/B tests may provide similar results without going too in-depth. Results are always quicker to generate and can be carried out by all sizes of websites from small blogs to e-commerce giants.

Aside from these drawbacks, multivariate can still provide meaningful results. As the only way to analyse the smaller details, it’s an essential tool for those looking to take an already engaging site and improve it further. It may be more expensive, but the results can be used in future campaigns, as well as for other pages on the site. Once you’ve realised the winning combination of elements, this will likely be transferable to all your other marketing efforts. Give multivariate testing a go and you could be impressed with the results.


Author Gyles Seward

Gyles is our Managing Director and knowledgeable in all things WordPress.

More posts by Gyles Seward