Running Headline and Image A/B Tests

Product Guides Engagement
1

A story that is underperforming might in fact be of interest to readers but merely require a makeover of the title or image. The Heads Up Display allows you to run A/B tests on title text and images in order to find out. Conduct A/B tests on text and images right within the Heads Up Display. Get results with clear winners for actionable insights into the best possible headlines and images for story links. Find the no-code setup instructions here.

Run as many variants of a test as desired. Use different headline and image combinations, or keep the images the same throughout to only test the titles and vice versa. See where tests are ongoing at a glance by the blue colored labels with the letters A/B.

The blank canvas is a thing of the past. Marfeel Copilot is integrated into the HUD to generate headline suggestions in any language based on what’s worked before on your own home page. The more tests you run, the better the suggestions will be.

Test Headlines

Testing headlines is as simple as clicking on one of the suggestions from Copilot. This will automatically create an A/B test comparing the original headline to the suggested one, which you can edit as much as you want.

Marfeel Docs - HUD & AB Testing Screenshots (14)
Marfeel Docs - HUD & AB Testing Screenshots (14)1440×900 86.9 KB

To test any other headline:

  1. Click on New A/B test
  2. Add new text
  3. Click Add variant to test additional titles
  4. Check the HUD for a preview of how the new title would look in context and adjust accordingly.
  5. Click Run test

Run test
Run test1440×900 81.9 KB

Test Images

Test images
Test images1440×900 79.4 KB

  1. Open a new A/B test
  2. Click the plus sign on the image to add a new image
  3. Upload a new image from your computer
  4. Slide the bar back and forth to crop and adjust
  5. Make sure to respect the original aspect ratio as indicated by the highlighted box
  6. Leave the title as-is to only test the image, or edit the text as well
  7. Click Save and Run test

Results

Return to the article detail window to find the test results for the corresponding link. Find the results for each variant.

Testing metrics

  • CTR: the Viewable CTR for that variant
  • Impressions: the number of users who viewed that variant
  • Clicks: the number of clicks on that test variant
  • CTR lift: the difference in CTR between the original and winning variant

Test in progress
Test in progress1440×900 79.2 KB

Check the HUD toolkit for all ongoing and archived A/B test results:

Marfeel Docs - HUD & AB Testing Screenshots (18)
Marfeel Docs - HUD & AB Testing Screenshots (18)1440×900 91.2 KB

Original won

You got it right the first time. If a new title or image doesn’t significantly increase the viewable CTR, then it may be that the topic itself is not interesting to concurrent users. Consider moving the article down the page or removing it completely to make room for more impactful stories.

No winner

Insufficient traffic across test variants or lack of a statistically important difference between them will generate this result. Marfeel will only declare a winner when there is 95% statistical confidence or higher.

Winner found

Well done, you’ve found an optimization opportunity! See how much the new version out-paced the original in terms of viewable CTR. Replace the original title and/or image.

Update content with winning results

The best practice for replacing the current headline or image with the new winner is to permanently change it in the CMS.

It’s also possible to set up an Experience that will automatically update the content with the winner. This is a convenient way to make sure that users will automatically see the winner, however best practice is still to go into the CMS at your convenience and change it from there.