Key takeaways
- Collect questions from where buyers already speak: sales calls, lost deals, RFPs and support, not from your keyword list or your positioning document.
- Group wordings into prompt families, place each family on the buying journey and give it a commercial weight before you read a single answer.
- Freeze the set and version it. A question set that changes every month cannot show you whether anything you did made a difference.
Every figure about your AI visibility rests on one decision made before any answer is read: which questions to ask. Choose them well and the figures describe your real position with real buyers. Choose them badly and you will measure, very precisely, something that does not matter.
The usual failure is not carelessness. It is asking the questions a brand would like to be asked, in the language the brand uses about itself. Those questions tend to produce flattering answers and very little insight. This playbook sets out a method for building a question set that reflects how buyers actually ask, in five steps, with a worked example.
Step one: collect questions where buyers already speak
Do not start with a blank page or a keyword tool. Start with the places where buyers have already told you, in their own words, what they need to know.
- Sales call notes and recordings. The questions prospects ask in a first call are close to the questions they ask an assistant before it.
- Lost deal notes. Why did the buyer choose someone else? The answer often contains the question they were really asking.
- RFPs and security questionnaires. These are lists of buyer constraints written by the buyer. Each requirement can become a question.
- Support and onboarding tickets. Early tickets reveal what new users assumed, and what they wished they had checked first.
- Community threads and review sites. How do practitioners in your category describe the problem to each other?
- Your own site search and search console data. Useful as a cross-check, but treat it as a supplement. People type fragments into a search box and full sentences into an assistant.
Aim for a long list first, without filtering. Fifty to a hundred raw questions is a healthy start for most B2B categories. Keep each one in the words it arrived in.
A useful test. If a question contains a word only your marketing team uses, it is your question, not the buyer's. "Which platforms offer unified decision intelligence?" is a positioning line. "Which analytics tools can our finance team use without SQL?" is a buyer.
Step two: group wordings into prompt families
Many raw questions are the same question asked differently. "Best analytics for regulated reporting", "Which BI tools handle FCA reporting?" and "What analytics platform should a UK fintech use for compliance reports?" share one intent. We call that group a prompt family, and each wording within it a prompt version.
Families matter for two reasons. First, they stop a single intent being counted many times over just because you found many ways to phrase it. Second, they are the honest unit of measurement. Answers to wordings of the same question tend to resemble one another, so ten answers from one family tell you much less than ten answers from ten families. Reading an AI visibility figure honestly explains why that matters when you come to read the numbers.
Within each family, keep two to four wordings that a real buyer might plausibly use. Vary the constraints a little (company size, sector, region) where buyers genuinely vary them, but do not invent constraints nobody asks about.
Then group families into scenarios: the broad situations a buyer is in. "Regulated reporting", "Switching vendors" and "Integrations" are scenarios. They make the set easier to read and help you see where you have gaps.
Step three: place each family on the buying journey
A buyer asks different questions at different points. We use four stages.
Problem. The buyer is naming the problem, not yet the solution. "How do fintechs keep regulatory reporting audit ready?"
Solution. The buyer knows the kind of product they need. "What kinds of analytics platform suit regulated reporting?"
Evaluation. The buyer is comparing named options. "Which analytics platforms should a UK fintech shortlist for regulated reporting?" or "What are the alternatives to Pennant Metrics for a mid-size team?"
Decision. The buyer is checking a specific vendor before committing. "Does Larchmont Data support UK data residency?" or "What do teams say about migrating to your brand?"
A balanced set covers all four, but most of the commercial weight usually sits at evaluation and decision. Problem stage questions are worth keeping, because they show whether you are connected to the problem before a buyer has named any vendor, but they are less likely to produce a qualified recommendation on their own.
Include questions that name competitors. "Alternatives to the market leader" and "Pennant Metrics versus Larchmont Data" are questions buyers ask constantly, and they are among the most revealing to measure.
Leave out questions that name you in a loaded way. "Why is your brand the best choice for fintechs?" tells you nothing about how buyers find you. Decision stage questions about you are fine, as long as they are the neutral checks a buyer would actually run.
Step four: weight by commercial value
Not every question deserves equal attention. Before you read any answers, agree with sales a commercial weight for each family: how much it would matter to the business to be put forward for it.
Keep it simple. A scale from one to ten works. The principle is that the weight comes from the business, not from the results. If you set weights after seeing the answers, you will be tempted, consciously or not, to weight up the questions where you already look good.
Weights let you report a qualified recommendation rate that reflects commercial reality. Being put forward for the question your best deals start from should count for more than being put forward for a question nobody buys from.
| Scenario | Prompt family | Stage | Example wording | Weight |
|---|---|---|---|---|
| Regulated reporting | Best platform for regulated reporting | Evaluation | Which analytics platforms should a UK fintech shortlist for regulated reporting? | 9 |
| Regulated reporting | Audit-ready analytics | Problem | How do fintechs keep regulatory reports audit ready? | 5 |
| Switching vendors | Alternatives to the market leader | Evaluation | What are the alternatives to Pennant Metrics for a team of 200? | 8 |
| Integrations | Warehouse integrations | Solution | Which analytics tools integrate natively with our data warehouse? | 6 |
| Integrations | Vendor integration check | Decision | Does your brand have a native connector for our warehouse? | 4 |
Step five: freeze it, version it and leave it alone
Once the set is agreed, freeze it. Give it a version, record the date and the exact wording of every prompt version, and keep it unchanged from one round of measurement to the next.
This is the step teams are most tempted to skip, and it is the one that makes everything else worth doing. The whole point of measuring is to compare. If the questions change between March and June, a change in your figures could be the result of your work or of the new questions, and you will never know which. Comparing like with like on the same questions is the difference between evidence and anecdote.
Questions will need to change sometimes. A new competitor arrives, a regulation changes, you launch into a new sector. When that happens, change the set deliberately, give it a new version and treat the first round on the new set as a fresh baseline. Do not edit a question quietly and carry on comparing.
Enough prompt families to say something about each scenario that matters, and few enough that you will keep asking them. For most B2B categories that means dozens of families rather than a handful, with two to four wordings each. If a figure rests on only a few families, read it as a direction, not a measurement.
What to do on Monday
- Book an hour with sales. Ask for the questions prospects asked in their last ten first calls, and the reasons in their last five lost deals.
- Pull your last three RFPs or security questionnaires. Turn each requirement that buyers would plausibly ask an assistant into a question.
- Write the raw list in buyer words. Strike out anything containing your own jargon.
- Group into families and scenarios. Keep two to four wordings per family.
- Place each family on a stage and agree weights with sales. Do this before you read any answers.
- Freeze version one. Date it, store it and agree that it will not change without a new version.
Then, and only then, start reading the answers.
The buyer-question prompt library
Thirty-two example prompt families across the four stages of the buying journey, with fill-in wording, constraints and a weight column, as a spreadsheet and a short guide.
Free with your work email
Request the download