About this channel
Google Search Central focuses on how websites are discovered, processed and understood in search. Crawling, indexing, site structure and Search Console recur throughout the selection, alongside…
Google Search Central focuses on how websites are discovered, processed and understood in search. Crawling, indexing, site structure and Search Console recur throughout the selection, alongside questions about AI and the ways websites are now being built. The videos approach search as a collection of concrete technical and editorial decisions that site owners need to understand.
Some episodes begin with a tightly defined question, such as whether sitemaps still matter or whether a site's own search-result pages should be blocked. Others explain how to read an indexing report. These topics give viewers a way to connect an individual setting or report with the larger process of making a website available to search engines.
The formats also include update roundups, technical analysis and Japanese-language office hours. A session using HTTP Archive and BigQuery looks at robots.txt files on a large scale, while other material examines SEO for sites built with AI assistance. Questions about Markdown and the influence of AI on search broaden the discussion from familiar technical controls to changing publishing and development practices.
Website owners, developers and people responsible for SEO are the clearest audience. Someone investigating an indexing issue can focus on the relevant report or crawling topic; someone reviewing how they build a site can look at the discussions of AI-assisted development. The channel offers explanations and questions that help connect site maintenance with search visibility. Its practical value lies in making specific concepts easier to examine before deciding what a particular website needs.
Latest episode
Do sitemaps still matter?
Sitemaps have been a core element of web development and search engine optimization for two decades, yet Search Console warnings like "Couldn't fetch" continue to generate concern among site owners…
Sitemaps have been a core element of web development and search engine optimization for two decades, yet Search Console warnings like "Couldn't fetch" continue to generate concern among site owners. Are XML sitemaps still necessary with modern search crawlers, or can search engines and AI bots figure out site structure on their own? In this episode of Search off the Record, Google Search Relations team members Martin Splitt and John Mueller break down the history, technical specifications, and some misconceptions surrounding sitemaps, RSS feeds, LLMs.txt, canonicalization signals, and Search Console indexing behaviors.
In this episode, you’ll learn: The Origins of Sitemaps: How early web crawling challenges led to the sitemap standard and how building a sitemap generator started John Mueller's career at Google.
Obsolete vs. Active XML Fields: Why Google dropped support for the priority and changefreq tags, and how the lastmod date is actually evaluated.
XML Sitemaps vs. RSS Feeds: How RSS feeds function as lightweight sitemaps for recent updates and why AI training crawlers utilize both.
Technical Limits & Index Files: The exact URL (50,000) and file size (50MB uncompressed) limits, alongside best practices for sitemap index files and robots.txt declarations.
HTML Sitemaps & LLMs.txt: Why user-facing HTML maps and Markdown-based LLMs.txt files cannot replace structured XML sitemaps for search engines.
Troubleshooting "Couldn't Fetch": How host load management and perceived site quality (crawl demand) trigger non-technical fetching errors in Search Console.
Key takeaways for SEOs & developers: Keep Sitemaps Enabled by Default: Even for small websites or dynamic CMS setups, keeping auto-generated sitemaps active carries no downside and assists with discovery as sites grow.
Provide Accurate lastmod Timestamps: Avoid setting all page timestamps to the current date; Google evaluates date reliability and will ignore lastmod signals if they are inaccurate or abused.
Submit Canonical URLs Only: Always place clean, canonical versions of URLs inside your sitemap file rather than parameter-tagged tracking URLs to help guide Google’s canonical selection.
"Couldn't Fetch" Isn't Always a Syntax Error: If Search Console reports "Couldn't fetch" on a valid, accessible XML file, the cause is often host load throttling or low crawl demand linked to perceived site quality.
Chapters 00:00 – Introduction & Welcome
00:57 – Sitemaps History & John Mueller's Path to Google
03:42 – Why Search Engines Started Using XML Sitemaps
04:46 – Deprecated Fields: Priority, Change Frequency, and lastmod
06:36 – Sitemaps for Small Sites vs. High-Volume News Outlets
08:50 – Canonical Selection & Parameterized URLs in Sitemaps
11:13 – Comparing RSS Feeds and XML Sitemaps
12:58 – Size Limits, Compression, Index Files, and robots.txt
14:15 – Custom File Naming, Privacy, and AI Crawlers
16:54 – Why HTML Sitemaps Don't Replace XML Sitemaps
19:06 – Will LLMs.txt Replace Sitemaps for AI?
22:05 – Decoding Search Console's "Couldn't Fetch" Error
24:16 – Hreflang, Image, and Video Sitemap Extensions
25:14 – Key Takeaways & Outro
Resources Mentioned:
Official Google Search Central Documentation: https://developers.google.com/search
Google Search Console: https://search.google.com/search-console
Don't forget to like and subscribe to the podcast on your favorite platform to catch every behind-the-scenes episode from the Search Relations team!
Episode transcript → https://goo.gle/sotr114-transcript
Listen to more Search Off the Record → https://goo.gle/sotr-yt
Subscribe to Google Search Channel → https://goo.gle/SearchCentral
Search Off the Record is a podcast series that takes you behind the scenes of Google Search with the Search Relations team.
#SOTRpodcast #SEO #GoogleSearch #SearchConsol#SEO #WebDevelopment #GoogleSearchConsole #Sitemaps #SearchEngineOptimization #GoogleSearchRelations #LLMstxt #TechnicalSEO #SearchOffTheRecord
Speakers: Martin Splitt, John Mueller
YouTube Shorts
Chinese SEO Office Hours
English/Hindi SEO Office Hours
Search Off the Record podcast
uploads
More episodes from this channel
Japanese Google Search Office Hours（ #Google検索オフィスアワー 2026 年 09 月 24 日）
24.9.2026
Google では、日本のサイト運営者のみなさまに向けて、2013 年より検索のご質問にお答えするオフィスアワーを定期的に行ってきました。幾度かのリブランディングを経て、2023 年 5 月より新しく Google 検索オフィスアワーとなりました。 Google 検索オフィスアワーでは、SEO 担当者、エンジニア、デザイナー、マーケティングなどサイトの運営に関わる皆さんからの、Google 検索に関するご質問に Google 社員がお答えします。 本オフィスアワーに関して、下記のブログ記事にて詳しくご紹介しています。 🔗 #Google検索オフィスアワー のご紹介 https://developers.google.com/search/blog/2023/04/introducing-japanese-office-hours?hl=ja 今後のイベントの予定は決まり次第こちらのページでお知らせいたします。 https://developers.google.com/search/events?hl=ja みなさまからのご質問、楽しみにしております！！ 🎥 過去の動画一覧 👉 https://goo.gle/search-oh-playlist 📄 質問フォーム 👉 https://goo.gle/search-oh
Should you block your Search result pages?
30.7.2026
Is your website’s internal search feature secretly acting as an open invitation for crawling lots and lots? In this episode of Search off the Record, Martin Splitt and John Mueller pull back the curtain...
How to read the Indexing Report
16.7.2026
Should you panic when your Search Console indexing report is showing pages that aren't indexed? Is a 404 error code always a sign of a broken website? In this episode of Search Off the...
Japanese Google Search Office Hours（ #Google検索オフィスアワー 2026 年 06 月 18 日）
18.6.2026
【本日のご質問】 6:53 マークアップでテキストを含む画像の使用 9:00 Google 検索のコア アップデート 11:10 重複コンテンツの処理 12:26 Discover のインプレッションが急落 13:52 バックボタンハイジャッキング 15:48 戻るボタンのバックボタンハイジャック 17:47 Search Console のデータ異常 19:34 今後の AI 検索に向けた情報発信の方針 22:18 primaryImageOfPage マークアップ 23:53 Search Console データの乖離と正規化の影響 27:09 コンテンツが Index...
Should I use markdown for my site?
15.6.2026
Should you convert your website into Markdown to help Large Language Models (LLMs) understand your content better? Is "llms.txt" worth the effort for SEO? In this episode of Search Off the Record, Martin Splitt...
Vibe Coding - yay or nay?
7.5.2026
In this episode of Search Off the Record, Martin Splitt and John Mueller from Google's Search Relations team dive deep into the world of AI-assisted development. They explore the reality of "Vibe Coding", the...
How AI Is Changing Google Search and SEO
1.5.2026
In this episode of Search Off the Record, Martin speaks with Nikola Todorovic (director of Software Engineering at Google Search) about how AI is changing Google Search. They discuss the evolution from traditional search...
Analysing Robots.txt at scale with HTTP Archive and BigQuery
23.4.2026
In this episode of Search Off the Record, Martin and Gary turn a simple robots.txt question into a data‑driven deep dive using HTTP Archive, WebPageTest, custom JavaScript metrics, and BigQuery. They explore how millions...
Are websites getting “fat”? Page weight, HTML size & Googlebot limits explained
30.3.2026
In this episode of Search Off the Record, Gary and Martin dig into what “page size” and “page weight” actually mean for developers, users, and search engines. They discuss exploding web page sizes: median...
Google crawlers behind the scenes
12.3.2026
Developers often talk about Googlebot as if it were a single program you could just run as “googlebot.exe”, but that is not how Google’s crawling actually works. In this episode of Search Off the...
Japanese Google Search Office Hours（ #Google検索オフィスアワー 2026 年 03 月 05 日）
5.3.2026
【本日のご質問】 5:33 テキスト検索結果画像の制御 7:30 Search Console レポートで異なるステータス 8:40 リンク否認ツールを使用する際のプロパティ 10:06 トップではなく下層ページが上位表示される 11:40 「公開 URL をテスト」が利用できない 13:35 サイト内の全ページがインデックスされない 15:05 「クロール済み - インデックス未登録」 16:27 複数のカテゴリページが重複判定される 18:55 表現の揺れがある検索クエリの取り扱い 19:54 イベントの構造化データのグローバル対応 21:21 URL 設計 22:46 JavaScript SEO の基本を理解する...
How Browsers Really Parse HTML (and What That Means for SEO)
26.2.2026
Martin and Gary unpack how HTML parsing really works, why the HTML standard is so lenient, and how messy markup can silently break key SEO signals like hreflang and rel=canonical. They revisit validators and...
Do You Still Need a Website in 2026?
12.2.2026
In this episode of Search Off the Record, Martin and Gary from the Google Search Relations team tackle a deceptively simple question: do you still need a website in 2026? Starting from the recurring...
Crawling Challenges: What the 2025 Year-End Report Tells Us.
3.2.2026
Join Martin and Gary as they dive into Search Off the Record's Episode 103, unpacking the 2025 Year-End report on crawling issues. Discover fascinating insights on faceted navigation, action parameters, irrelevant parameters and more,...
SEO, AIO, GEO, your site, & third-party support to optimize for LLMs
8.1.2026
In this episode, John Mueller and Danny Sullivan continue their conversation about the changing landscape of Search. They discuss the practicalities of hiring an SEO professional in the age of AI: Do you really...
Japanese Google Search Office Hours（ #Google検索オフィスアワー 2025 年 12 月 18 日）
18.12.2025
Google では、日本のサイト運営者のみなさまに向けて、2013 年より検索のご質問にお答えするオフィスアワーを定期的に行ってきました。幾度かのリブランディングを経て、2023 年 5 月より新しく Google 検索オフィスアワーとなりました。 Google 検索オフィスアワーでは、SEO 担当者、エンジニア、デザイナー、マーケティングなどサイトの運営に関わる皆さんからの、Google 検索に関するご質問に Google 社員がお答えします。 本オフィスアワーに関して、下記のブログ記事にて詳しくご紹介しています。 🔗 #Google検索オフィスアワー のご紹介 https://developers.google.com/search/blog/2023/04/introducing-japanese-office-hours?hl=ja 今後のイベントの予定は決まり次第こちらのページでお知らせいたします。 https://developers.google.com/search/events?hl=ja みなさまからのご質問、楽しみにしております！！ 🎥 過去の動画一覧 👉 https://goo.gle/search-oh-playlist 📄 質問フォーム 👉 https://goo.gle/search-oh
Thoughts on SEO & SEO for AI, part 1
17.12.2025
In this episode, John Mueller and Danny Sullivan talk about SEO and the new kids on the block: AIO, GEO and others. Danny shares how these things interact with each other and what he...
Analyzing performance on Google Search
4.12.2025
The Search Console performance reports are powerful tools to help you analyze your site’s presence and results on Google. It provides a wide range of data and analytical capabilities to help you get the...
How Search Off the Record tackles SEO and web development
3.11.2025
Celebrate the 100th episode of Search Off the Record with Martin, Lizzi, Cherry, John and Gary as they revisit memorable moments, touching on a bunch of topics. This special episode touches on Google's mobile-first...
Japanese Google Search Office Hours（ #Google検索オフィスアワー 2025 年 09 月 25 日）
25.9.2025
【本日のご質問】 4:08 「クロールの統計情報」で提示される URL 6:08 head 内に記述された HTML 要素 7:38 サイト移転に伴いドメインの評価は分散するか 9:23 クロールリクエスト数が激減 10:44 検索アナリティクスの平均順位 12:04 サイトが検索結果に表示されない 13:44 セキュリティ警告への対応 17:03 作品名より略称で検索した方が上位表示される 【関連リンク：今回ご紹介した動画や記事】 Search Central Live Tokyo: The 2025 Return https://developers.google.com/search/blog/2025/09/scl-tok?hl=ja サイトを移転する方法 https://developers.google.com/search/docs/crawling-indexing/site-move-with-url-changes?hl=ja 表示回数、掲載順位、クリック数とは https://support.google.com/webmasters/answer/7042828?hl=ja...
Continue reading
You may also like
Innohub TV
Watch this content also on Innohub TV
We picked clips from Innohub TV that continue the same topic.
