Digital.gov Guide

Testing and assessment

Methods to validate your design and content to ensure they meet user needs.
Illustration of two people pointing at a gauge that measures customer satisfaction
Digital Gov Logo

Multivariate tests

On this page

    Method: Multivariate tests

    What

    A multivariate test tests variations to multiple sections or features. It is different from an A/B test, which tests variation to a single section or feature.

    Why

    Multivariate tests help you find which combination of design changes improves performance the most.

    How to do it

    1. Identify the call to action, content section, or feature that needs to be improved to increase conversion rates or user engagement.
    2. Develop a list of possible issues that may be hurting conversion rates or engagement. Specify in advance what you are optimizing for. This can be done through design hypotheses or another method.
    3. Design several solutions that aim to address the issues listed. Test different combinations of design changes to see which performs best.
    4. Use a web analytics tool that supports multivariate testing to set up the testing environment. Conduct the test for long enough to produce statistically significant results.
    5. Analyze the results. Determine which solution produced the best conversion or engagement rates. Review the other solutions to see if there is information worth examining in future studies.

    Time required

    2 to 5 days of effort over the course of 1 to 4 weeks

    Qualitative data Metrics