Experiment types

An overview of the types of experiments available in Optimize

Optimize offers a diverse range of experiments designed to suit the needs of your business. This article explains how A/B, redirect, and multivariate (MVT) tests work and offers examples of how to use them.

A/B tests

An A/B test is a randomized experiment using two or more variants of the same web page (A and B). Variant A is the original. Variants B through n each contain one or more elements that are modified from the original (for example, a different colored call-to-action button). In some experiments, variant B may be a completely different version of a web page.

Each variant is served at similar times so that its performance can be observed and measured independent of other external factors. You can also limit your experiment to a specific audience with targeting. Optimize captures the performance data for each variant against an objective and identifies the leader.

Example

An A/B test of two variants of a product page (A and B). Variant A has a blue photo, while variant B has a red photo.

Learn more about how to create an A/B test

Multivariate tests (MVT)

A multivariate test (MVT) tests variants of two or more elements simultaneously to see which combination creates the best outcome. Instead of showing which page variant is most effective (as in an A/B test), MVT identifies the most effective variant of each element as well as analyzing the interactions between those elements. MVT tests are useful for optimizing multiple aspects of a landing page, for example.

Example

You can test two headlines (original "H1" and variant "H2") and three hero images (original "A" and variants "B" and "C") simultaneously, which would produce:

  • 2 sections (the headline and the images), with
  • 3 and 2 variants (respectively), for a total of
  • 6 combinations (the number of variants multiplied by each other, or 3 x 2)

Redirect tests

A redirect test, (a.k.a. split URL test), is a type of A/B test that allows you to test separate web pages against each other. In redirect tests variants are identified by URL or path instead of an element(s) on the page. Redirect tests are useful when you want to test two very different landing pages, or a complete redesign of a page.

Example

Test two different landing pages, with different URLs:

  • Original - www.example.com/landing1
  • Variant - www.example.com/landing2

Test a redesigned page hosted on a subdomain:

  • Original - www.example.com
  • Variant - new.example.com

Learn more about how to create a redirect test

Help