Agent Skills: llm-crawler-access-check
Check whether a website's robots.txt allows the AI crawlers that decide visibility in ChatGPT Search, Perplexity, Claude, Gemini, and Microsoft Copilot. Use when someone asks whether AI bots are blocked, whether to allow or block GPTBot, why a site never appears in AI answers, or wants a robots.txt review for AI crawlers. Reads only robots.txt, then returns a per-agent allow/block table, the exact rule responsible for each verdict, and the precise lines to change. Distinguishes training crawlers from the search crawlers that actually control citations.
UncategorizedID: davepoon/buildwithclaude/llm-crawler-access-check
3,515524
Install this agent skill to your local
Skill Files
Browse the full folder contents for llm-crawler-access-check.
Loading file tree…
Select a file to preview its contents.