Content analyzer
What do noindex and nosnippet do (and how do they affect AI citations)?
noindex and nosnippet are 2 meta robots directives with opposite scopes: noindex removes a page from all search engines entirely; nosnippet blocks only the text extracts that engines show and AI systems quote. Either 1 will silently kill AI citations on an otherwise well-optimized page — noindex by removing the page, nosnippet by blocking the quotable passage.
Seeklyn checks for noindex and nosnippet in the "AI access" crawl-gate group on every live-URL content report — run your page free at /dashboard/content.
What do noindex and nosnippet do?
noindex and nosnippet are meta robots directives (or X-Robots-Tag HTTP headers) that instruct search engines how to handle a page. noindex tells every engine to remove the page from its index: the page cannot rank, cannot appear in AI Overviews, and cannot be fetched as a citation source. nosnippet allows a page to remain indexed (and ranking) but prohibits the engine from extracting and displaying text — which is exactly what AI answer engines do when they cite a passage.
The distinction matters for AEO: a page with noindex is invisible to all engines. A page with nosnippet may still appear as a blue link in classic search results but will never be quoted in an AI answer. Both directives are set via the same mechanism — a content attribute in the HTML meta robots tag or the X-Robots-Tag HTTP response header — so a misconfigured deployment can accidentally block citations without anyone noticing.
How does Seeklyn detect them?
Seeklyn's live-URL analysis fetches the page and parses both the HTML meta robots tag and the X-Robots-Tag header for noindex and nosnippet tokens. max-snippet:0 is treated as equivalent to nosnippet because it sets the maximum text extract length to 0 characters — functionally identical to blocking snippet extraction entirely. Both checks appear in the 'AI access' audit group with evidence grade strong and source 'Google Search Central'.
A noindex result is a fail (not a warn) in the crawl-gate audit because the page is invisible to all engines — there is no partial state. A nosnippet result is a warn rather than a fail because the page can still rank and be found; it is only the quoting step that is blocked. The fix for both is to remove the directive from the meta tag or HTTP header.
What are common accidental causes?
Most noindex and nosnippet blocks are accidental, introduced at deployment rather than intentionally. A staging environment noindex directive not removed before launch is the most common cause — whole domains go live blocked. A CMS setting like 'discourage search engines' (WordPress) sets noindex site-wide. Aggressive snippet-blocking added to prevent scraping via max-snippet:0 also blocks AI Overviews. A CDN or edge layer that injects X-Robots-Tag headers for non-HTML content types can bleed into HTML pages.
Seeklyn flags noindex as a hard fail and nosnippet as a warn so both are visible in the report rather than passing silently. Because the crawl gate is checked before passage scoring, a noindex page shows all 3 crawl-gate checks clearly, making it easy to spot that the entire analysis is moot until the directive is removed.
Frequently asked questions
Does nosnippet affect regular Google ranking?
No — nosnippet does not affect ranking or indexing, only text extraction. The page can still rank position 1 on Google. But AI Overviews, Perplexity, and ChatGPT all rely on text extraction to generate citations, so a nosnippet page will not be quoted in AI answers even if it ranks well.
Can I block some snippets but not AI citations?
Not precisely — nosnippet is a broad directive that blocks all snippet extraction, including AI citations. You can set max-snippet to a positive value (e.g. max-snippet:200) to allow some text extraction while limiting very long excerpts, which is a middle ground between full snippets and full blocking.
Is X-Robots-Tag more powerful than the meta tag?
Both are equivalent in authority — search engines respect whichever is more restrictive. An X-Robots-Tag noindex header overrides a meta tag that says index, and vice versa. Seeklyn checks both sources when analyzing a live URL.