Test Five Ad Headlines Before You Pay for Ads

The standard way to test ad headlines is to run them all and let the ad platform pick a winner. Each headline needs enough impressions and clicks to reach a result, and you pay for every impression, the losing headlines included.

A simulated panel gives you a cheaper first cut. You show five headlines to up to 1,000 simulated people from your target audience, see which ones they pick and why, and send only the top two to a paid test.

1. Write five headlines for one offer. Keep the offer, price and image the same. Change only the headline. Here are five for a meal-kit brand:
   1. “Dinner in 20 minutes, no grocery run”
   2. “Five new recipes on your doorstep every week”
   3. “Cook restaurant meals on a Tuesday”
   4. “Spend less on takeout this month”
   5. “Fresh ingredients, measured for you”
2. Describe your audience. For example: “US adults aged 25 to 45 who cook at home three or more nights a week and order takeout at least once a week.” HumanPanel splits the audience into worlds, such as busy parents, young professionals and budget-minded couples.
3. Ask four questions.
   1. Which of these headlines would make you stop scrolling? (single choice)
   2. Which headline would you be least likely to click? (single choice)
   3. What does your first pick promise you? (free text)
   4. After reading your first pick, how likely are you to visit the website? (single choice: likely, unsure, unlikely)
4. Run the survey on 200 or more respondents. With four worlds, 200 respondents gives each world around 50 people, enough to compare rankings between worlds.

Start with the overall ranking, then open each world. Check whether the overall winner also wins in each world. Look for three patterns:

• One winner everywhere. Send the top headline and the runner-up to your paid test.
• Different winners in different worlds. Match headlines to ad sets. If busy parents pick “Dinner in 20 minutes” and budget-minded couples pick “Spend less on takeout”, give each headline its own targeting.
• A heavy “least likely” vote. Drop any headline which more than a third of a world picks as least likely to click.

Then read the free-text answers to question 3. If people describe a promise you didn’t intend, such as “cheap food” for a premium brand, the headline sets the wrong expectation, even when the headline wins.

Put the top two headlines into a paid split test with the same budget, audience and creative. You now pay to test two strong options instead of five untested ones.

Record the simulated ranking next to the paid results. After three or four tests, you will see how closely the simulated rankings match your real click-through rates, and how much weight to give them next time.

• Simulated people read your headline in a survey, not in a busy feed. Attention, design and placement are missing from the test.
• Rankings hold up better than percentages. Treat “Headline C won 38% of first picks” as a ranking, not a forecast of click-through rate.
• Small gaps are noise. With 200 respondents, treat a gap under 7 points between two headlines as a tie.

Write the survey for free, then run the test when you’re ready. Most runs cost $5 to $10, and you see the maximum cost before you start. New accounts get $1 of free credit. Sign up with your email.