The URL in question has a noindex directive in the HTTP response HTML alone, without it being present in the rendered HTML post-JavaScript execution.
Why is this important?
Rendering webpages consumes considerable resources and time as compared to quickly fetching the raw HTML content.
Google employs a dual-stage crawling approach: initially, Googlebot examines the source HTML, followed by page rendering for a closer look at the rendered HTML before updating its index.
Understanding this 'Processing' phase can be intricate, as there are concurrent processes. Google determines pages for rendering based on the result of checking meta robots tags.
As per the guide by Google, Understand the JavaScript SEO basics:
"Googlebot queues pages for rendering, unless instructed otherwise by a robots meta tag or header to not index the page."
Essentially, noindex directive leads to no rendering. This means these pages won't be lined up for rendering, thus Googlebot fails to see the lack of noindex in the rendered content, resulting in pages not being indexed.
Moreover, the discrepancy can create confusion as to whether pages are meant to be indexed or remain nonindexed, which might lead to unintended noindex status.
What does the Optimization check?
This Optimization is activated for any URL that has a noindex directive in the source HTML but is absent in the rendered HTML.
Examples that trigger this Optimization
The Optimization triggers for URLs with a meta robots=noindex directive in the <head> of the source HTML that isn't carried over to the rendered HTML.
For instance, consider this source HTML:
<!doctype html><html lang="en"><head> <meta name="robots" content="noindex,follow"> ...</head><body>...</body></html>
Contrasted with this rendered HTML:
<!doctype html><html lang="en"><head> <meta name="robots" content="index,follow"> ...</head><body>...</body></html>
How do you resolve this issue?
This issue is deemed 'Critical' as it can significantly affect your site's organic search visibility. It is imperative to address such vital problems with urgency.
Begin by determining whether these pages should be indexed. Subsequently, make sure the robots directives before and after JavaScript rendering align.