Rehearse Your Interview Guide on Simulated People

Most interview guides get tested on the first real participant. Halfway through the session you notice a leading question. A prompt gets a one-word answer. A follow-up you needed never gets asked. The participant took two weeks to recruit, and you spent the hour on repairs.

HumanPanel gives you a place to rehearse first. You pick simulated people who match your participants, put your questions to them in a chat, and read how each person responds. Every problem you fix here is one you won’t hit with a real participant.

1. Create the people. Make an audience from a plain description of your participants, for example “operations managers at logistics firms with 50 to 500 staff, who approve software purchases.” An audience holds up to 20 people across up to 3 worlds. Or reuse the people from any finished survey run.
2. Pick who joins. Start a chat and choose up to 20 people. Pick a spread of ages, roles and levels of interest in your topic.
3. Ask your guide in order. Send each question as written. Every person in the chat answers every message, so one pass through a 12-question guide with ten people gives you 120 answers to read.

Each simulated person has a background, beliefs, blind spots and a set level of knowledge about your subject. Someone who has never heard of your category says so. Read the replies for four problems.

• Leading questions. If most people accept the premise and repeat your wording back, the question is doing the work for them. Rewrite “How much time does manual reporting waste each week?” as “Walk me through how you report on shipments each week.”
• Dead-end prompts. Short, flat replies such as “Yes, we use spreadsheets” mean the prompt invites a yes or no. Turn closed questions into requests for a story or an example.
• Missing follow-ups. When answers raise a topic your guide never returns to, such as a budget freeze or a security review, add a probe for the topic.
• Jargon. If people ask what a term means, or answer a different question from the one you asked, expect real participants to stumble on the same word.

For focus groups, press “Let them talk among themselves” after a prompt. The people in the chat respond to each other instead of to you. You see which topics start disagreement and which get no response, and where a moderator needs to step in.

Use the rehearsal to plan timing. A topic where eight people agree in one round needs less time in the real session than a topic where the group splits.

Simulated people answer in full sentences and stay on topic. Real participants go quiet, change the subject and contradict themselves. A rehearsal tests your questions. The rapport you build in the room stays untested.

Treat the answers as a check on your guide, not as findings. Bisbee et al. (2024) found AI answers varied less than real people’s and shifted with small wording changes. Expect a wider spread of views in real sessions than in the rehearsal.

Take the guide for your next study, create an audience from your screener criteria, and run the first five questions past ten people. Fix what breaks, then book the real sessions. New accounts get $1 of free credit, and writing surveys is free. Sign up with your email.