Media
5 items
Description
Discover a site's full sitemap, page URLs, links, and exportable datasets in the background — submit a domain, it keeps running. Sitemap, Backlink & Dataset Generator discovers every page on a website — its sitemap, individual page URLs, and internal links — and runs the job entirely in the background. Enter a domain, close the popup or side panel, and it keeps working; come back later to download the results. HOW IT WORKS The extension first checks the site's robots.txt for a published sitemap and expands it fully, including nested sitemap indexes. If a site has no sitemap at all, it automatically falls back to crawling the site's actual pages by following on-site links — the same discovery approach used by search engines, done politely: it respects robots.txt rules, honors crawl-delay settings, and backs off automatically if a site signals it's being rate-limited. Sites that explicitly disallow crawling in robots.txt are skipped, not routed around. WHAT YOU GET - A clean list of every discovered page URL, automatically filtered to exclude ads, tracking links, binary assets (images, PDFs, scripts), and admin/login/cart/checkout pages — or customize your own exclude list. - Download results as a plain text file, or (Pro) as a validated, Google-compatible XML sitemap, automatically split into multiple files with a sitemap index if the site is large. - Quick Dataset mode: pull a clean URL dataset from a single sitemap or domain fast, without saving a full job — copy it or download it in seconds. - Page Capture: save the actual text content (and images) of a page you're viewing — one page at a time, added by you or automatically as you browse a site you've allowed — into a growing dataset. No page design, no crawling beyond pages you yourself visit. Free plan: up to 2 pages. Pro: unlimited, plus a clean Markdown export with each page's source attached. - Files tab: every download organized into folders by site, with search/filter and a one-click preview of any dataset in a new tab. FREE PLAN - Crawl depth up to 2, up to 10 pages per site - One new site every 10 minutes - Up to 10 sites saved - Page Capture: up to 2 pages - .txt downloads only PRO PLAN ($9/month) - Unlimited sites, unlimited crawl depth and pages - No cooldown between jobs - Page Capture: unlimited pages - Every export format: .txt, .xml, .jsonl, .md PRIVACY The extension only accesses domains you explicitly submit. Discovered page URLs stay on your device — they're never sent to any server. See the privacy policy link on this listing for full details.