Sitemap Builder

Crawls the site and generates a sitemap.xml that follows sitemaps.org, with the last-modified date when the site sends one, for sites that don't produce one.

Sitemap Builder

What it does

Runs a focused crawl of the site and produces a sitemap.xml that follows sitemaps.org. It starts from the home page (or the URL you give), follows the internal links breadth-first and lists every page that loads, at its final address after redirects: pages that answer with an error, files that are not pages and links to other sites stay out. Each URL carries <lastmod> when the site sends a last-modified date. No <priority> or <changefreq>: search engines ignore them.

It gives the same file as the free Sitemap Builder with the same limits (the free page stops at 40 pages).

When to use it

  • When the audit flags that key pages never get cited by LLMs, a likely symptom of discovery failure.
  • On custom sites (non Shopify/WooCommerce/Wordpress) without an automatic sitemap.
  • After major URL restructurings.

Required inputs

  • store_id (required): per-store metering, 3 runs/month.
  • start_url (optional): where the crawl starts; without it, the store's home page.
  • max_pages (optional, default 100, at most 500), max_depth (optional, default 3, at most 6).

Example output

{
  "start_url": "https://acme.it/",
  "sitemap_xml": "<?xml version=\"1.0\" encoding=\"UTF-8\"?>...",
  "pages_count": 100,
  "hit_page_cap": true,
  "hit_deadline": false
}