By SocialCRM · Sources checked September 7, 2026
Updated September 7, 2026: ChatGPT search access and model-training access are separate decisions. Earlier guidance on this page incorrectly treated GPTBot as the crawler shops needed for ChatGPT search.
Use the correct crawler for the task
OpenAI identifies OAI-SearchBot as its search crawler. GPTBot crawls content that may be used for training foundation models. ChatGPT-User handles some user-initiated requests and does not determine search inclusion.
A shop can allow OAI-SearchBot while disallowing GPTBot. Choose training permission deliberately. Do not change that permission merely to pursue search visibility.
Inspect the rules that actually apply
Start with the current robots.txt response on the preferred website host. Check the named user-agent groups and the wildcard group. A missing OAI-SearchBot group is not proof of a block; a wildcard rule may allow access.
If a developer adds a named group, review the complete resulting policy. Do not accidentally remove restrictions for private or irrelevant routes. Robots rules guide cooperative crawlers; they are not authentication controls.
- Identify the public service and location pages that should be accessible.
- Check the effective rule for OAI-SearchBot on those paths.
- Check that the pages load without login or unintended indexing restrictions.
- Ask the hosting provider to review legitimate crawler requests against published OpenAI IP ranges.
- Record the date, inspected URLs, applicable rules, and unresolved hosting checks.
Do not confuse a test with verified access
Sending a request with an OAI-SearchBot user-agent string can reveal some server behavior. It does not prove that genuine requests from OpenAI can pass hosting or security controls. Record that distinction in the result.
Likewise, an allowed robots rule establishes only the policy observed at the time of the check. A crawler visit does not prove that a page appeared in an answer. Verify answer observations separately.
Follow access work with content and evidence
Check that the public pages contain the correct services, location details, and approved qualifications. Use direct internal links so visitors can reach the relevant facts. Then collect a bounded set of answers and compare claims with dated sources.
Keep consumer-interface observations separate from provider API responses. Each can support a useful check, but they describe different contexts. Use the AI visibility guide below for a repeatable review process.
Sources and further reading
Platform guidance is linked below. The workflows and study design are SocialCRM recommendations, not ranking guarantees.
Put the guidance to work
Start with a recorded check of what AI says about your shop. Compare the answer with verified facts before deciding what to change.
Start your AI Shop Checkup