An e-commerce site with several thousand references is launching a new product category. The product pages are online, the internal linking is in place, and yet Google takes weeks to discover these pages. The problem does not stem from the content or the links, but from a poorly configured XML sitemap that sends crawlers into dead ends.
When the XML sitemap sends the wrong signals to search engines
The same scenario often occurs: an automatically generated sitemap file by a CMS that lists all the site’s URLs indiscriminately. Redirected pages, URLs with sorting parameters, non-canonical versions, everything is included.
The concrete result: Google and Bing crawlers waste their crawl budget on unnecessary pages instead of exploring the content we want to index. A sitemap that contains URLs returning 301 or 404 codes is a sitemap that confuses rather than guides.
To observe this phenomenon, one can examine the sitemap of the Bargento site: it illustrates the logic of a file that targets only the pages intended to appear in search results, without pollution from technical URLs.
The operational rule can be summed up in one sentence: each URL in the sitemap must return a 200 code and correspond to the canonical version of the page. If a URL is blocked by robots.txt, redirected, or has a noindex tag, it has no place in the file.
Lastmod tag in the sitemap: useful signal or unnecessary noise
The lastmod tag indicates to engines the last modification date of a page. On paper, it’s a lever to speed up the re-crawling of updated content. In practice, many sites make it useless by updating it with every regeneration of the sitemap, even without any actual content modification.
Google only utilizes this tag when the date is consistently and verifiably accurate. If the engine finds that the lastmod changes while the content remains the same, it eventually ignores this information for the site in question.
Making lastmod reliable in practice
The lastmod date must correspond to a real modification of the page content, not just a simple re-save in the back office. On WordPress, some sitemap plugins update the lastmod as soon as a tag or associated category is modified, without touching the text. This behavior degrades the reliability of the signal.
The right approach: condition the update of the lastmod to an actual change in the page body (text, images, structured data). Feedback varies on this point depending on the CMS, but the logic remains the same everywhere.
Changefreq and priority tags: what Google ignores in your sitemap file
Many sitemaps are still filled with changefreq and priority tags, sometimes configured carefully. However, Google’s official documentation is clear: these two tags are ignored by the engine.
In concrete terms, assigning a priority of 1.0 to your homepage and 0.3 to your blog articles does not change the crawl order or the frequency of exploration. The time spent configuring these values would be better invested in three concrete actions:
- Ensure that each URL in the sitemap returns an HTTP 200 code and is neither redirected nor blocked
- Make sure all URLs point to their canonical version (no duplicates with/without trailing slash, with/without www)
- Establish a reliable lastmod, correlated to actual content changes
A sitemap stripped of these unnecessary tags is more readable, lighter, and sends no false signals.
XML Sitemap and IndexNow: two complementary mechanisms for SEO
The sitemap provides a structured and permanent view of the site’s URLs. Engines consult it at their own pace, according to their own crawl schedule. For sites that frequently publish or update content, this delay can be problematic.
This is where IndexNow comes into play. This protocol allows you to directly notify certain engines of the addition, modification, or deletion of a URL. Bing recommends using both together: the sitemap as a reference for overall coverage, IndexNow as real-time notification.
Concrete use case
A news site publishes about twenty articles a day. Its XML sitemap is updated every hour. With IndexNow enabled, each publication triggers a ping to Bing (and compatible engines) without waiting for the sitemap to be re-crawled.
Google does not support IndexNow to date, but it offers an indexing API for certain types of content. Combining XML sitemap, IndexNow, and indexing API covers the widest spectrum of engines and use cases.
Sitemap and generative search: an evolving role
Engines are increasingly integrating AI-based features into their results. A publication from the Bing team dated July 2025 presents sitemaps as a signal of URL coverage for AI-based search experiences.
This does not guarantee that a URL will be selected to feed a generative response. The sitemap remains a discovery tool, not a ranking lever. Its role in this context is to ensure that crawlers explore all relevant content on the site.
- The sitemap aids coverage: it signals the existence of a page, not its quality
- Content, internal linking, and thematic relevance determine whether the page will be retained
- A clean sitemap prevents a generative AI crawler from indexing technical or outdated pages instead of high-value content
The structure of the sitemap remains a technical prerequisite. What makes the difference in results is what the pages it references contain. A clean file, limited to canonical URLs with a reliable lastmod, constitutes the minimum foundation to remain visible in an increasingly complex search ecosystem.
