XML Sitemaps Explained (and How to Get Yours Right)

An XML sitemap is a file listing the URLs you want search engines to know about, along with optional hints like when each page last changed. It doesn't guarantee indexing, but it helps engines discover pages efficiently — especially on large sites or those with pages that aren't well linked internally.

What belongs in a sitemap

Include canonical, indexable URLs that return a 200 status and that you actually want in search. Leave out the clutter:

• No redirected URLs (3xx) or error pages (4xx/5xx).
• No pages blocked by robots.txt or marked noindex.
• No non-canonical duplicates — list the canonical version only.
• No parameter-laden tracking URLs.

The lastmod hint

The lastmod date tells engines when a page genuinely changed, which can help them prioritize recrawling. It only helps if it's accurate — stamping every URL with today's date on every build trains engines to ignore it. Reserve lastmod for real content changes.

Size and splitting

A single sitemap can hold up to 50,000 URLs and must stay under 50MB uncompressed. Larger sites split URLs across multiple sitemaps referenced by a sitemap index file. Splitting by section also makes it easier to spot indexing problems in one area of the site.

Submitting and referencing it

Reference your sitemap in robots.txt with a Sitemap line, and submit it in your search engine's webmaster tools. Both methods help; together they ensure the sitemap is discovered and monitored.

Keep it clean over time

Sitemaps drift. Pages get deleted, URLs change, sections get retired. A sitemap full of redirects and dead URLs wastes crawl effort and muddies your indexing reports. Regenerate it as part of your release process so it always reflects the live, canonical state of the site.