Key takeaways
- ChatGPT and Claude recommend from what they can find and read. Both search the web for some questions and not for others, and both show the sources they used when they do search.
- Start with access. OpenAI and Anthropic each document a search crawler that is separate from their training crawler, and blocking one is not the same as blocking the other.
- Then work the method on each assistant separately: measure your buyers' questions, read what each answer rests on, close the evidence gaps, and measure the same questions again.
A B2B tech company gets recommended more often by ChatGPT and Claude when the public evidence connecting it to a buyer's question is easy for each assistant to find, read and cite. In practice that means letting each company's search crawler reach your site, publishing specific pages that answer the questions buyers actually ask, earning independent sources that say the same thing, and measuring the result on each assistant separately. Nobody outside OpenAI and Anthropic controls what their assistants recommend, so this work improves the evidence an answer can draw on; it does not decide the answer.
This playbook sets out the method we use, step by step, with a worked fictional example and the limits of each step. It sits beside two other pieces and does not repeat them. The first ninety days covers the order and timing of a first quarter of work, and what a brand can influence in an AI answer draws the line between what is yours to change and what is not. This page starts where they stop: with what each assistant's maker says about how it searches and shows its sources, and what that means for the work.
How ChatGPT searches and shows its sources
OpenAI's help article Searching the web with ChatGPT says ChatGPT may search the web automatically when a question would benefit from current information, and that a person can also start a search by hand. Responses that use web search may include citations, and a Sources view lists the cited sources alongside other relevant links. The same article says ChatGPT search sometimes works with other search providers, and that when it does, it usually rewrites the question into one or more targeted searches. It also says plainly that search results and citations can be incomplete, out of date or wrong.
For site owners the article is direct. It says ChatGPT ranks search results using several factors, that placement is not assured, and that to make a website eligible for inclusion you should allow OpenAI's search crawler, OAI-SearchBot, and make sure your host or content delivery network lets through traffic from OpenAI's published search crawler addresses.
OpenAI's developer page Overview of OpenAI Crawlers describes the crawlers themselves. OAI-SearchBot is used to surface websites in ChatGPT's search features, and the page says sites that opt out of it will not be shown in ChatGPT search answers, though they can still appear as navigational links. GPTBot is a different robot, used to crawl content that may be used to train OpenAI's models. The page says each robots.txt setting is independent of the others, so a site can allow OAI-SearchBot while disallowing GPTBot. A third agent, ChatGPT-User, visits pages when a person's request in ChatGPT calls for it; the page says it does not crawl the web automatically and is not used to decide whether content may appear in search. For search, OpenAI says a robots.txt change can take about a day to take effect.
Last checkedHow Claude searches and shows its sources
Anthropic's help article Enable and use web search says that when a question touches on topics that benefit from current information, Claude uses a search tool to ground its answer in content from the live web, and that responses which search include citations so a reader can check the sources. On Team and Enterprise plans an owner has to switch web search on for the workspace first. In the newer Claude experience there is no switch at all: Claude searches when it judges that searching helps. A person can also ask Claude outright to search, or not to.
Anthropic's developer documentation for the web search tool is more specific about when that happens. It says Claude decides when to search based on the request. It searches when the answer depends on information that is current, changing or outside its training data, including information about specific organisations or products that might have changed. It answers directly, without searching, when a request draws on stable knowledge. Citations are always on when that tool searches.
Anthropic's help article on whether it crawls the web and how site owners can block the crawler names three robots. ClaudeBot collects content that may be used for training. Claude-User fetches pages when a person asks Claude a question, and the article says blocking it may reduce a site's visibility for user-directed web search. Claude-SearchBot indexes content to improve search results, and the article says blocking it may reduce a site's visibility and accuracy in those results. Anthropic says its robots honour robots.txt, and that blocking them by IP address may not work reliably, because it stops the robots from reading the robots.txt file at all.
Last checkedWhat the two have in common, and where they part
Read side by side, the two sets of documentation point to four things a B2B tech company can act on.
- Search is not always on. Both assistants search for some questions and answer others from what the model already holds. A question about which vendors suit a buyer's need is the kind both describe searching for, because it concerns products that change. An answer given without search draws on training material fixed at an earlier date, and nothing you publish this month reaches it.
- Search and training are separate switches. Each company runs one robot for training and another for search. A site that blocked "AI crawlers" with one broad rule may have shut out search as well as training, without meaning to. Check which robots your robots.txt names, and what your host or firewall blocks.
- Sources are shown, and they are the evidence you can read. When either assistant searches, its answer carries citations. Those citations are the most direct view you will get of what an answer rested on.
- They draw on different searches. OpenAI says ChatGPT search works with other search providers; Anthropic describes its own search tool. Nothing in either set of pages says the two find the same pages, so a page one assistant cites may never reach the other. Treat them as two separate audiences.
None of the pages above asks for special markup, a special file or a particular format. What they ask for is access. The rest is the evidence itself, and that is what the method works on.
The method, step by step
We run the same five steps for every company we work with, on every assistant we measure: measure the questions, read the evidence, find the gaps, act, and measure again. Each step below names what goes in, what comes out, and what the step cannot tell you.
A worked example runs through every step. It is fictional: a UK HR software company that sells to employers of 200 to 2,000 staff, whose strengths are payroll integration and absence management. It tracks two rivals, which we will call Rival A and Rival B.
Step 1: Measure the questions buyers ask, on each assistant
Build a set of questions in the words your buyers use, freeze it, and ask each question of ChatGPT and of Claude several times, recording every answer with its date. Building a buyer-question set sets out how to collect, weight and freeze the questions; the point here is to ask each one of each assistant, separately, and more than once.
Repeat runs matter because answers change from one run to the next. A single answer that names you, or leaves you out, is an anecdote. For each answer, record whether you were named, whether your own pages were cited, and whether you were put forward for the buyer's need. Mention, citation and recommendation explains why those three outcomes are always kept apart.
- Inputs: a frozen question set, the rivals you track, and access to each assistant with its search available.
- Outputs: a dated record of every answer, per assistant, with the three outcomes counted over their own base.
- What this cannot tell you: why an answer came out the way it did, or what a particular buyer saw. When we collect answers, we do it through each company's API with that company's own search, and that is not the same as what the ChatGPT or Claude app shows a person signed in to their own account.
In the example. The HR company freezes 15 questions, among them "Which HR software suits a UK employer of 500 staff that needs payroll integration?" and "What are the alternatives to Rival A for a mid-sized UK company?". Each is asked of each assistant twice. On the payroll question, the company is named in 1 of 2 ChatGPT answers and in 0 of 2 Claude answers. Rival A is put forward in both ChatGPT answers.
Step 2: Read what each answer rests on
For every answer that searched, open the citations and read them. Note who owns each source (you, a rival or an independent site), what kind of page it is, and which claim in the answer it supports. Do this for ChatGPT and for Claude separately, because they draw on different searches.
Look at the answers that did not search as well. An answer with no citations was written from what the model already holds. It shows how the assistant describes you from that, which is useful, but it cannot be traced to a page.
- Inputs: the recorded answers and their citations.
- Outputs: for each answer, a list of the sources it rested on, by owner and by kind, with the claim each one supported.
- What this cannot tell you: everything the assistant read. A citation shows what was used and shown, not every page the search returned or the model weighed. Nor can it show what training material shaped an answer that did not search.
In the example. On the payroll question, both ChatGPT answers cite a comparison article on an independent HR blog and Rival A's payroll integration page. The HR company's own site is not cited in either. Claude's answers cite a different pair of sources: a UK software directory listing and Rival B's pricing page.
Step 3: Find the evidence gaps
For each answer where you were absent, or named but not put forward, write down what was missing. Was there no page of yours on the question at all? Was there a page, but without the specific fact the buyer asked about? Did the rival have an independent source saying the same thing while you had none? Could the assistant's search crawler reach your page in the first place? Evidence gaps describes the twelve classes we sort these into, with the first fix for each, so they are not repeated here.
Then add the check that the documentation above makes specific to these two assistants: confirm that OAI-SearchBot, Claude-SearchBot and Claude-User are allowed in your robots.txt, and that your host or firewall does not block them.
- Inputs: the source lists from step 2, your own pages, your robots.txt and your firewall settings.
- Outputs: a list of gaps per question and per assistant, each tied to the answers it came from.
- What this cannot tell you: whether closing a gap will change the answer. A gap is a reading of what was missing, not proof of what the assistant needed.
In the example. The HR company finds three gaps on the payroll question. Its payroll integration is described only inside a product tour that loads by script, so there is no page a crawler can read on the question. Rival A's page names the UK payroll systems it connects to, and the HR company's material names none. And the independent comparison article does not mention the HR company at all. The access check finds a fourth problem: a firewall rule added a year earlier to stop scraping blocks unrecognised crawlers, and both search crawlers are among them.
Step 4: Act on pages, independent sources and facts
Close the gaps in order: the questions that matter most commercially first, and within them the gaps that recur across answers. The work falls into three kinds.
- Pages. One page per priority question, about that question, in the buyer's words, with the specific facts an answer would need. The text sits in the page itself, not in images or behind scripts.
- Independent sources. For each claim that matters, at least one place outside your site that states it accurately: a partner listing, a directory profile, a correction sent to the author of a comparison. You cannot write these, but you can supply accurate material.
- Facts. The specific, checkable details: the systems you integrate with, the standards you meet, the size of company you suit. Stated the same way everywhere they appear.
Log each change with its date, the gap it closes and the questions it should affect. That log is what lets step 5 connect a change in the answers to a piece of work.
- Inputs: the gap list, a content owner, and someone who can change the website and its access settings.
- Outputs: published pages, requested or updated independent sources, corrected access settings, and a dated log of every change.
- What this cannot tell you: whether either assistant has found the change yet. Publishing a page is not the same as having it read.
In the example. The HR company fixes the firewall rule first, because nothing else counts while both search crawlers are shut out. It then publishes a page titled with the payroll question, naming the six UK payroll systems it connects to, with a short paragraph on each. It updates its listing on the UK directory that Claude cited, so the listing names the same systems in the same words, and sends the author of the comparison article a short note linking to the new page. Each change goes into the log with its date.
Step 5: Measure the same questions again
Once the changes have been live long enough to be found, ask the same frozen questions again, of each assistant, the same number of times, and record the answers the same way. Compare like with like: the same questions, the same assistants, the same counting. If anything about the question set or the collection changed, say so, because the comparison is weaker for it. Reading an AI visibility figure honestly covers how much weight a change on a small number of answers can bear.
- Inputs: the frozen question set, the baseline record and the change log.
- Outputs: a before and after per assistant and per question, with the questions that did not move reported as plainly as those that did, and the next round of gaps.
- What this cannot tell you: that your work caused the change. A provider can change its model or its search at any time, and rivals publish too. The change log makes the connection plausible; it does not prove it.
In the example. On the re-measure, the new payroll page is cited in 1 of 2 ChatGPT answers to the payroll question, and the company is named in both. Claude's answers now cite the updated directory listing and name the company in 1 of 2, as an alternative to Rival B. The question about alternatives to Rival A has not moved on either assistant. The company reports both results side by side, and puts the alternatives question at the top of the next round's gap list.
What this cannot promise
The method is honest about its edges, and anyone who sells it to you should be too.
- Answers vary from run to run. Even when nothing has changed, the same question can return different names on different days. A few answers either way is not a result, which is why the method asks each question more than once and compares like with like.
- No one outside OpenAI and Anthropic controls their assistants. They decide when their assistants search, how they rank what they find, and what their models already hold, and they can change any of it without notice. Work on evidence improves what an answer can draw on; it does not decide the answer.
- There is no reliable timing evidence for a new domain. We know of no published, reliable measure of how long a new page, or a new website, takes to appear in either assistant's answers, and we do not offer one. OpenAI's note that a robots.txt change takes about a day applies to its crawler settings, not to being recommended. The first ninety days sets out a sensible order for the work without pretending to know the date it pays off.
- Training material is out of reach. An answer that does not search draws on what the model learned earlier. Nothing published today reaches it until the provider trains again, if it ever does.
So a fixed date by which you will be named, or a set number of recommendations, is something no one outside the two providers can honestly offer.
If you want this run for you
The five steps can be run by hand on a small question set. If you would rather have them run for you, the AI visibility audit puts your buyer questions to the assistants we agree with you, maps the evidence gaps and gives you a prioritised plan, and AI visibility measurement and marketing repeats the measurement on the same questions and does the work that closes the gaps.
