Noindex URLs in a Sitemap: Why the Signals Conflict

A sitemap that lists a page while the page says noindex is sending two different messages. The sitemap says the URL is important enough to submit for discovery, while the page itself tells supported search engines not to keep it in search results.

That conflict is usually a configuration mistake, especially when it appears after a CMS migration, staging launch, SEO plugin change, or bulk template edit.

Use the Sitemap URL Checker to sample sitemap URLs and detect noindex in both HTML meta tags and X-Robots-Tag response headers.

What noindex actually does

Google documents noindex as an indexing rule that can be delivered in two main ways:

• a robots meta tag in HTML, such as <meta name="robots" content="noindex">;
• an HTTP X-Robots-Tag response header containing noindex.

Google says that when Googlebot crawls an accessible page and sees the rule, the page is dropped from Google Search results.

That is different from robots.txt. A robots.txt rule controls crawling. A noindex directive controls indexing after the crawler can access and read the directive.

Why a noindex URL should usually not be in a sitemap

Google's sitemap guidance says to include the URLs you want to see in search results. If a URL is intentionally noindex, it normally does not belong in that list.

For example, these pages may intentionally be excluded from indexing:

• internal search results;
• account or login pages;
• filtered faceted URLs;
• thank-you pages;
• duplicate print views;
• staging or test content;
• thin utility pages that are not meant for organic search.

If those URLs are intentionally noindex, remove them from the sitemap rather than removing the noindex rule solely to make the checker green.

The dangerous case: accidental noindex

The more serious scenario is when the page should be indexed but inherits a noindex directive by mistake.

Common causes include:

• a WordPress or CMS “discourage search engines” setting left enabled after launch;
• an SEO plugin applying noindex to a content type or taxonomy;
• a sitewide HTTP header added at the server or CDN layer;
• a template component that injects <meta name="robots" content="noindex">;
• staging rules copied into production;
• environment variables that still identify production as a preview environment.

In that situation, removing the URL from the sitemap would hide the symptom rather than solve the indexing problem. First decide whether the page is supposed to be indexable.

How SitemapChecker.tools detects noindex

The live URL audit checks three places:

1. <meta name="robots" ...>
2. <meta name="googlebot" ...>
3. the X-Robots-Tag response header

A noindex found in any of these locations is surfaced as an error for a sitemap URL because it conflicts with the usual purpose of sitemap inclusion.

The checker does not pretend to reproduce Google's complete indexing pipeline. It reports the directives returned by the live HTTP response it receives.

Important: robots.txt can hide a noindex rule from crawlers

Google's noindex documentation contains an important warning: for a noindex rule to be effective, the crawler must be able to access the page and see it.

If robots.txt blocks the page, Google may never retrieve the HTML or response header containing the noindex directive. That means a combination of “blocked by robots.txt” and “noindex” is not a stronger exclusion strategy; it can actually prevent Google from seeing the noindex rule.

If your goal is to keep a page out of search results, use the indexing controls Google documents for that purpose rather than assuming a robots block and noindex reinforce each other.

How to fix noindex URLs in a sitemap

First classify each affected URL.

The page should not be indexed

Keep the intentional noindex and remove the URL from the sitemap generator.

Then check that related internal systems are not re-adding it during each sitemap rebuild.

The page should be indexed

Investigate where the noindex comes from before changing the sitemap.

Check:

• page source for robots meta tags;
• response headers for X-Robots-Tag;
• CMS visibility settings;
• SEO plugin templates;
• server and CDN header rules;
• deployment environment settings.

After removing the accidental noindex, fetch the page again and confirm that the rule is gone.

What about nofollow?

nofollow is not the same as noindex.

A page can contain nofollow without instructing Google to remove that page from search results. The checker specifically treats noindex and none as indexing exclusions. It does not flag every robots directive as a sitemap error.

What if the page is not HTML?

The HTTP X-Robots-Tag can apply to non-HTML resources such as PDFs. That is why checking only HTML meta tags is incomplete.

If a sitemap includes a non-HTML URL and the response sends X-Robots-Tag: noindex, the indexing conflict still exists even though there is no HTML <head>.

Verify the fix

A practical workflow is:

1. Run the sitemap through the URL checker.
2. Identify URLs marked noindex.
3. Decide whether each URL is intentionally excluded.
4. Remove intentional noindex URLs from the sitemap.
5. Remove accidental noindex directives from pages that should rank.
6. Re-run the live audit.
7. For Google-specific confirmation, use Search Console URL Inspection.

The goal is not to maximize the number of URLs in the sitemap. The goal is to keep the sitemap focused on the preferred, crawlable URLs you actually want indexed.