How HumanPanel Helps Product Research

Most product decisions get made without research, because real research is slow and expensive for the question at hand. Recruiting a panel and fielding a survey can take days or weeks. So teams guess, ship, and learn the hard way.

HumanPanel fills the gap before fieldwork. You run your survey on a panel of simulated respondents in minutes, usually for $2 to $5. You get a directional read on how different kinds of people might answer, and why.

It does not replace talking to real customers. It helps you walk into that conversation with sharper questions and fewer blind spots.

A run takes your questions and your target audience, and returns a report with results split by segment. The steps look like this:

1. Write your survey. Up to 20 questions, using single choice, multiple choice, yes/no, true/false or free text.
2. Describe who you want to hear from. For example, “UK parents of school-age kids who shop online weekly.”
3. HumanPanel designs the “worlds.” These are 1 to 10 segments of your audience, such as city centre, suburbs and small towns. Each segment carries about 14 weighted attributes, including age, income, education, politics and media habits.
4. Respondents are seated. Up to 1,000 simulated people are placed across the worlds in proportion. Each has a background, beliefs and blind spots, plus a profile with age, occupation and personality traits.
5. Answers stream in live. Every respondent explains the reason behind each answer.
6. You get the verdict. Results overall and by world, recurring themes with verbatim quotes, and a written summary. It is also emailed to you.

Respondents are chosen because they fit your audience, not because they are likely to like your idea. A panel that only flatters you is useless.

Pricing is pay-as-you-go. Most runs cost $5 to $10, you see the maximum cost before you start, and you are only charged what the run actually used.

The best use is the early, messy questions you would otherwise answer by gut feel.

1. Concept tests

Describe the product idea in two or three sentences, then ask whether people would use it and what worries them. The free-text reasons are often more useful than the percentages. They show objections you had not thought of.

2. Pricing checks

The Van Westendorp price sensitivity method uses four questions: too cheap, a bargain, getting expensive, and too expensive. That fits inside one run. Splitting results by world shows whether suburban buyers react differently from city buyers.

3. Feature prioritisation

Ask respondents to pick the three features they would miss most, using multiple choice with a maximum of three. Compare the picks across segments. A feature that wins in one world and flops in another is a positioning question, not just a roadmap one.

4. Messaging and naming

Test two headlines, taglines or product names on the same simulated population. HumanPanel can reuse a saved audience, so both versions are judged by exactly the same people. That makes the comparison fairer than two separate panels.

5. Piloting your real survey

Before paying for real fieldwork, run the questionnaire on HumanPanel first. Confusing wording, missing answer options and leading questions tend to show up in the reasons respondents give.

When an answer surprises you, you can chat with that respondent and ask follow-up questions, much like a short interview.

Academic work on AI respondents is promising but mixed, and you should know both sides before relying on the results.

The encouraging side. In Argyle et al. (2023), researchers gave GPT-3 thousands of demographic backstories from real US survey participants. Its answers tracked the response patterns of many real subgroups closely, which the authors call “algorithmic fidelity.” Brand, Israeli and Ngwe found that willingness-to-pay estimates from GPT-3.5 were realistic in size and matched a human consumer study.

The cautionary side. A later version of the Brand, Israeli and Ngwe paper reports that LLM willingness-to-pay estimates are sometimes comparable to human ones but often inaccurate, and occasionally point the wrong way. The earlier version also found that fine-tuning did not improve how well models captured differences between customer segments.

Bisbee et al. (2024) found that ChatGPT’s average opinion scores matched a major US election survey closely. But answers varied less than real people’s did, shifted with small wording changes, and changed when the same prompt was rerun three months later.

What that means for you. Treat synthetic results as a fast first read, not a final answer. They are best at surfacing reasons, objections and wording problems, and weakest when you need precise numbers or fine differences between groups. HumanPanel labels every summary as simulated for this reason.

Use HumanPanel before real research, not instead of it. A simple loop looks like this:

1. Explore. Run a quick concept or pricing survey to find the obvious objections and the segments that react differently.
2. Refine. Fix confusing questions, drop weak ideas and sharpen the ones that survived. Rerun on the same saved audience to compare.
3. Validate. Take the refined survey to real customers or a real panel. You now spend your fieldwork budget on better questions.

The cost of step 1 is small enough to do it for every idea, not just the big ones.

Try it. Sign up at humanpanel.ai with just your email. You get $1 of free credit and no credit card is needed, and writing surveys is always free.