Dashboard

How to Prompt AI to Write a Customer Survey

Models write surveys that quietly assume the answer you want. Here is the two-pass method that catches it, plus the scale rules nothing applies on its own.

Steve Jefferson
Steve Jefferson
Developer Advocate
6 September 20261 min read

Ask a model to write a customer survey and it will hand you eight questions that all quietly assume the answer you want. "How much did our new dashboard improve your workflow?" is not a question, it is a press release with a rating scale attached. Prompting AI to write a customer survey well is mostly about forcing it to produce neutral wording, then auditing what it produced for the bias it added anyway.

Here is the two-pass method, the audit prompt that catches the problem, and the rules about scales that no model will apply unless you tell it to.

Pass one: brief for the decision, not the topic

The single biggest quality jump comes from telling the model what decision the survey informs. A survey about "customer satisfaction" produces generic questions. A survey about "whether to keep the free tier" produces useful ones.

I need a customer survey. Context:

Decision it informs: whether to keep our free tier or replace it
with a 14-day trial.
Audience: 400 current free-tier users, mixed technical level.
Length limit: 6 questions, under 3 minutes.
What I already know: 4% convert to paid; most never invite a
teammate.
What I must not do: signal that we are considering removing the
free tier.

Write 6 questions. For each one, state in a single line what
decision-relevant thing it measures. If a question does not change
what I would do depending on the answer, do not include it.

That final constraint deletes about half of what models normally produce. Every question should have an answer that changes an action, and if you cannot name the action, the question is decoration that costs you completion rate.

Pass two: audit the questions for bias

Do not ask the same model in the same breath to critique what it just wrote enthusiastically. Start a fresh pass with the questions pasted in and a specific checklist:

Audit each question below for survey bias. Check for:

1. Leading wording (assumes a positive or negative experience)
2. Double-barrelled questions (asks two things, allows one answer)
3. Loaded terms (words that carry judgement: "helpful", "problem",
   "improved")
4. Assumed behaviour (asks about frequency of something the
   respondent may never do)
5. Unbalanced scales (more positive options than negative)
6. Missing escape option (no "not applicable" or "have not used it")

For each issue, quote the exact phrase and give a neutral rewrite.
If a question is clean, say so and move on.

Splitting generation and critique into two passes matters more than the checklist itself. A model asked to check its own fresh output tends to defend it, which is why prompting AI to check its own work is a separate technique rather than an extra sentence.

The scale rules a model will not apply on its own

State these explicitly or you will get inconsistent scales across a single survey.

Rule

Why

Same scale direction throughout

Mixing "1 is best" and "5 is best" corrupts your data and nobody notices until analysis

Balanced positive and negative points

Three positive options and two negative ones inflates your result by design

Label every point, not just the ends

"3" means different things to different people; "Neither easy nor difficult" does not

Odd number of points only when neutral is a real answer

For "did this work", neutral is usually a dodge

Never combine a rating and an open box in one question

People answer one and skip the other

Add a plain instruction that no question may use the words "helpful", "easy", "improved", "better" or "problem" in its stem. Those five words carry most of the bias in most surveys.

Open questions: exactly one, at the end

One open-ended question outperforms three. The one to use is some version of "What nearly stopped you from using this?" or "What did you expect to happen that did not?", because both surface specifics rather than sentiment.

Put it last. Open boxes early in a survey depress completion of everything after them.

When the responses come back, the analysis is a different prompting job with different failure modes, and summarising customer feedback with AI covers the part where a model quietly averages away the one furious response that mattered.

Test it on the model before you test it on people

Before sending, run a cheap dry run: ask the model to answer the survey three times in character as three different customer profiles you describe, then read the answers.

You are not learning anything about your customers. You are finding questions that are impossible to answer, ambiguous, or that produce the same response no matter who is asked. A question that gives identical answers across three very different personas is not measuring anything. This is a lighter version of roleplaying as a customer, used as a proofreading tool rather than a research method.

What not to delegate

The sample and the send are yours. Which 400 people get it, whether responses are anonymous, what you promise to do with the answers, and whether you tell people the survey is closing all shape your response rate more than the wording does.

And the interpretation stays yours too. A model reading your results will find themes, and it will find them whether or not they are there. The broader craft of getting precise output rather than agreeable output is covered in prompt engineering, and it applies here more than almost anywhere, because a badly worded survey does not fail loudly, it just returns confident numbers that are wrong.

For the underlying question design principles, the Pew Research Center's write-up on questionnaire design is a better reference than anything a model will summarise for you. When the goal shifts from gathering feedback to announcing something publicly, the prompting approach changes too, which is covered in how to prompt AI to write a press release.

FAQ

How many questions should a customer survey have?

Six or fewer for an email survey to existing users. Every question past six costs completion rate, and the questions that get dropped are the ones at the end, which are usually your open ones.

Should I let AI write the invitation email too?

Yes, and give it the same treatment: a bias audit pass, and a rule that it may not describe the survey as "quick" or "just two minutes" unless that is measured.

Can AI analyse the responses as well?

It can cluster and summarise, which is useful at volume. Keep the raw responses and read a random twenty yourself, because summaries lose the outliers that tend to matter most.

What is the single most common survey mistake?

Double-barrelled questions. "How satisfied are you with the speed and reliability?" cannot be answered by anyone who thinks one is fine and the other is not.

How did this land?

About the author

Steve Jefferson
Steve Jefferson

Developer Advocate

Steve builds something with Swarmz every week and writes up what worked, what broke, and what he'd do differently. Tutorials and hands-on guides are his lane.

Share

Get the next post in your inbox

One email a month. Product updates, engineering posts, and the best of Built with Swarmz.

I agree to receive emails about AI building tips and Swarmz product news. Unsubscribe any time.