Media
2 items
Description
Live robots.txt analyser — toolbar icon plus per-cluster status for 120+ AI, search and commercial crawlers. Robots.TXT Radar analyses the current site's /robots.txt in real time and shows an animated color-coded icon in your Chrome toolbar — automatically on every page load, no clicking required. 🟢 GREEN ✓ — path is allowed for crawling 🔴 RED ✗ — path is disallowed 🟢 TEAL ✓ — no robots.txt found, all paths open 📡 SCANNING — animated radar sweep while fetching HOW IT WORKS On every navigation Robots Radar fetches /robots.txt for that domain (cached for 1 hour per domain), parses it using Google's exact specification, and evaluates the current URL path against the correct user-agent group. BOT PRIORITY Rules are evaluated in this order, falling back to the next if no explicit group is found: Googlebot → Bingbot → CCBot → Google-Extended → * wildcard SPECIFICATION-COMPLIANT PARSING • Longest-path specificity rule (Google/RFC 9309 standard) • Allow beats Disallow on equal path length • Multiple groups for the same agent are merged • Agent normalisation: Googlebot/2.1 and googlebot* match correctly • Relaxed blank-line handling (Googlebot's documented behaviour) • Correct * and $ wildcard path matching FEATURES • Animated radar sweep while scanning — satisfying to watch • Dark mode / Light mode toggle, preference remembered • Per-domain caching — no redundant fetches • Click the toolbar icon for full details: – Which bot group was matched – The single winning rule with Allow/Disallow badge – Specificity bar showing match precision – Syntax-highlighted robots.txt source viewer PRIVACY Robots Radar only contacts the domain you are currently visiting to fetch its public /robots.txt file. No data is collected, stored, or shared. No analytics, no tracking, no third-party requests.