Quick A/B testing of a live website

Usually, 1 - 2 weeks and a sufficient number of website visitors are required to conduct an A/B test. Otherwise, if the sample is too small, there is a risk that false conclusions will be drawn.

But what should be done if there is not enough time or the website gets little traffic in general? Should testing and optimization be dropped altogether? No - in such a case Visual AI Testing is very handy.

For this we need the following two Google Chrome extensions:

1. Clueify Chrome Extension to take a screenshot of the website and have it analyzed directly by our AI
2. VisBug to quickly modify a live website without the need of programming skills

After you have installed both you can start with the A/B test.

Experiment

Let's assume we are working on a project for the streaming platform Disney+ and are a part of the design team. We are asked to decide whether the two CTAs on the US website should be blue or transparent with a border.

We could now take both of the design varations live and track the traffic on the site. This procedure will take a test time of at least two weeks and the data received through two weeks of analysis may still not be sufficient to have a sophisticated conclusions. Instead we can get crucial feedback in a few minutes through an analysis with the Clueify AI, which is train on data of more than 100.000 different people.

Step 1 - Control variation analysis

To analyze the control variation (also called the original variation) we just need to do the following:

1. Click on the Clueify Chrome Extension
2. Select "Visible" to limit the screenshot to the currently visible area
3. Optional - Enter a name for the analysis (e.g. "Control")
4. Optional - Add the analysis to a project (e.g. "Disney+")
5. Click "Analyze"

Once the analysis is complete, a new tab will open automatically and your results are shown in the Clueify WebApp. However, this analysis is only needed at a later stage in our comparison.

Step 2 - Creating and testing a variation

Since we have already tested our control variation, it is time to create another variation of our design. To do this, activate the VisBug extension that we installed earlier.

By clicking on the icon a toolbar should appear on the left side. Now select the "SIGN UP NOW" button and click on the paint bucket in the toolbar. Change the color to black to create a transparency effect.

To color the border of the button white we can use the tool with the pencil, which is located just below the paint bucket. Now repeat these two steps for the second CTA button as well.

Finally, we finished our second variation and can move on to testing.

Now we need to remove the toolbar by deactivating VisBug. Don't worry, your changes will be preserved and will disappear only when you reload the page.

To analyze the variation, use the Clueify extension again and proceed in the same way as in the first step.

Step 3 - Making a data-driven design decision

After analyzing both variations and marking the relevant area in the Clueify App, we can make a decision.

As shown in the result below in the control variation the two CTAs get 7% of the total attention. In contrast, in our variation, the attention rate is only 5%.

Hence, we have a clear winner. The Disney+ design team should stick with the blue buttons.