Lab

These pages exist to measure how search engines and AI assistants read Born in the Gap. They carry no policy content and are kept out of the sitemap on purpose.

What is being tested

• Open page: allowed for every crawler. A control.
• Closed to GPTBot: disallowed in robots.txt for OpenAI's training crawler only.
• Closed to ClaudeBot: disallowed for Anthropic's training crawler only.
• Closed to PerplexityBot: disallowed for Perplexity's crawler only.

A crawler that fetches the page it has been asked not to fetch is recorded, with the date. Only automated crawlers are logged, by their declared user agent; nothing about human visitors is recorded on these pages beyond the site's normal analytics.

Method and results are published by Modern Mouse, which built this site.