Agent Skills: llm-crawler-access-check
Check whether a website's robots.txt allows the AI crawlers that decide visibility in ChatGPT Search, Perplexity, Claude, Gemini, and Microsoft Copilot. Use when someone asks whether AI bots are blocked, whether to allow or block GPTBot, why a site never appears in AI answers, or wants a robots.txt review for AI crawlers. Reads only robots.txt, then returns a per-agent allow/block table, the exact rule responsible for each verdict, and the precise lines to change. Distinguishes training crawlers from the search crawlers that actually control citations.
ID: davepoon/buildwithclaude/llm-crawler-access-check
3,515524
Install this agent skill to your local
Skill Files
The files that make up llm-crawler-access-check.
More agent skills
- cal-com-automationAutomate Cal.com tasks via Rube MCP (Composio): manage bookings, check availability, configure webhooks, and handle teams. Always search tools first for current schemas.
- confluence-automationAutomate Confluence page creation, content search, space management, labels, and hierarchy navigation via Rube MCP (Composio). Always search tools first for current schemas.
- closing-costsCalculates itemized state-specific closing costs for mortgage refinance transactions across 10 licensed states, with product-specific fees for Conventional, FHA, FHA Streamline, VA IRRRL, and VA Cash-Out.
- tidyTriggered by "tidy up", "clean up transactions", "categorize uncategorized", "organize my transactions"
- figma-automationAutomate Figma tasks via Rube MCP (Composio): files, components, design tokens, comments, exports. Always search tools first for current schemas.
- macos-designDesign and build native-feeling macOS application UIs. Use this skill whenever the user asks to create a desktop app, macOS app, Mac-style interface, Apple-style UI, system utility, or anything that should look and feel like a native Mac application. Also trigger when users mention "native feel", "desktop app design", "Apple design patterns", "sidebar layout", "traffic lights", or want to build tools/utilities that feel like they belong on macOS. This skill covers layout, composition, interaction patterns, animations, light/dark mode, and all the subtle details that make an app feel like Apple built it.