Do sitemaps still matter?

Sitemaps have been a core element of web development and search engine optimization for two decades, yet Search Console warnings like "Couldn't fetch" continue to generate concern among site owners. Are XML sitemaps still necessary with modern search crawlers, or can search engines and AI bots figure out site structure on their own? In this episode of Search off the Record, Google Search Relations team members Martin Splitt and John Mueller break down the history, technical specifications, and some misconceptions surrounding sitemaps, RSS feeds, LLMs.txt, canonicalization signals, and Search Console indexing behaviors. In this episode, you’ll learn: The Origins of Sitemaps: How early web crawling challenges led to the sitemap standard and how building a sitemap generator started John Mueller's career at Google. Obsolete vs. Active XML Fields: Why Google dropped support for the priority and changefreq tags, and how the lastmod date is actually evaluated. XML Sitemaps vs. RSS Feeds: How RSS feeds function as lightweight sitemaps for recent updates and why AI training crawlers utilize both. Technical Limits & Index Files: The exact URL (50,000) and file size (50MB uncompressed) limits, alongside best practices for sitemap index files and robots.txt declarations. HTML Sitemaps & LLMs.txt: Why user-facing HTML maps and Markdown-based LLMs.txt files cannot replace structured XML sitemaps for search engines. Troubleshooting "Couldn't Fetch": How host load management and perceived site quality (crawl demand) trigger non-technical fetching errors in Search Console. Key takeaways for SEOs & developers: Keep Sitemaps Enabled by Default: Even for small websites or dynamic CMS setups, keeping auto-generated sitemaps active carries no downside and assists with discovery as sites grow. Provide Accurate lastmod Timestamps: Avoid setting all page timestamps to the current date; Google evaluates date reliability and will ignore lastmod signals if they are inaccurate or abused. Submit Canonical URLs Only: Always place clean, canonical versions of URLs inside your sitemap file rather than parameter-tagged tracking URLs to help guide Google’s canonical selection. "Couldn't Fetch" Isn't Always a Syntax Error: If Search Console…