Media
5 items
Description
See a page's JSON-LD errors, meta robots, X-Robots-Tag, canonical, hreflang, llms.txt and robots.txt rules for each AI crawler. Schema & AI-Crawler Inspector shows, in one click, how a page presents itself to search engines and AI crawlers. FREE, FOR THE PAGE YOU ARE ON • Structured data: every JSON-LD block, its @types, and errors and warnings for the types Google documents (such as Organization, Product and Article). JSON-LD added by the page's own scripts is included. • Indexing: meta robots, bot-specific meta tags (e.g. googlebot), the X-Robots-Tag response header, canonical (HTML or Link header) and hreflang alternates, with common mistakes flagged. • AI crawlers: whether robots.txt allows or blocks this page for the major AI crawlers (OpenAI, Anthropic, Perplexity, Google, Apple and others), with the exact rule that decided it. Bots whose operators say they may not follow robots.txt for user-triggered fetches are marked. • llms.txt: whether the site has one, and whether it follows the llms.txt format. PRO ($3/month or $24/year) • Bulk scan: the tabs you select, or up to 200 pasted URLs. • CSV export with a column per AI crawler (cells are protected against spreadsheet formula injection). • History of your scans, stored only in your browser; clear it any time. • Compare two scans of the same URL side by side. PRIVATE BY DESIGN The extension reads the page you open it on, plus that site's /robots.txt and /llms.txt, from your own browser and without cookies. It sends nothing to the developer: no analytics, no tracking, no remote code. It asks for access to other sites only when you start a Pro bulk scan, and only for the sites in your list and their subdomains (or all sites while a selected-tabs scan runs), and can give that access back when the scan ends. Payments for Pro are handled by ExtensionPay and Stripe on their own pages; the extension never sees your card. Support: brian@agentharbor.agency