Author: Ben

IndexNow is a free, open protocol that lets your site tell Bing, Yandex, Naver, Seznam, Yep and Amazon the moment a URL is added, updated or deleted — one ping reaches them all, and Google is the one engine that does not participate.

How IndexNow works — push instead of pull

Classic crawling is pull: a search engine visits your site on its own schedule and discovers changes eventually. IndexNow flips that to push: when you publish or update a page, your site sends a tiny notification containing the URL and a key that proves you own the site.

Important nuance: IndexNow is a notification, not a guarantee. It gets your URL in front of the crawler fast; whether the page is indexed still depends on quality — exactly as with Google. A thin page pinged a hundred times is still a thin page.

Who uses IndexNow — and who doesn’t

• Participating: Microsoft Bing, Yandex, Naver, Seznam.cz, Yep and Amazon (IndexNow FAQ); by 2023 over 60 million websites were using the protocol (Wikipedia). Submit to any one endpoint and the URL is shared with the others.
• Not participating: Google. Google tested the protocol years ago but still relies on its own crawling, sitemaps and Search Console. Don’t let anyone tell you IndexNow speeds up Google indexing.
• Why it still matters in 2026: Bing’s index powers Bing search and Microsoft Copilot, and it’s one of the web indexes AI assistants draw on. Faster in Bing can mean faster in AI answers — see how to get cited in AI search.

Set up IndexNow in 5 minutes

On WordPress (easiest)

• Official plugin: install IndexNow (by Microsoft Bing) from Plugins → Add New. It generates the key, hosts the key file and automatically submits every new, updated or deleted post. That’s what this site uses.
• Or your SEO plugin: Rank Math (Instant Indexing module), All in One SEO and Yoast SEO Premium include IndexNow support. Use one method, not several, to avoid duplicate pings.
• On Cloudflare? Its Crawler Hints feature can send IndexNow signals for you.

Any other site (manual)

• Create a key of 8–128 characters (letters, numbers, hyphens).
• Host it as a text file named {your-key}.txt in your site root, containing the key.
• Submit changed URLs to api.indexnow.org/indexnow — one URL via a GET request, or up to 10,000 URLs in a single JSON POST.

Rules that keep you out of trouble

• Only submit real changes. New pages, meaningful updates and deletions. Re-pinging unchanged URLs wastes your quota and signals nothing.
• Keep the key file reachable. If a security plugin or firewall blocks it, every submission fails verification.
• Don’t replace the basics. IndexNow complements your XML sitemap and internal links — it doesn’t replace them. Bing still uses both.
• Deleted a page? Submit it too, so engines drop it faster instead of showing a dead result.

Related guides: Bing Webmaster Tools: complete setup guide · The Google Indexing API

FAQ

How do you get cited by AI search? Make sure AI crawlers can index the page, then write the clearest sourced answer to one specific question: AI Overviews, ChatGPT, Copilot and Perplexity only cite pages they can find in their index, and in tests sourced statistics and quotations raised visibility by up to 40%.

How does AI search decide which pages to cite?

AI search tools answer from pages they retrieved from a search index, then cite the pages they used. Google is explicit that AI Overviews and AI Mode draw on pages that are indexed and eligible to be shown in Google Search with a snippet. ChatGPT search, Microsoft Copilot and Perplexity work the same way with their own crawlers and indexes.

That gives two filters. First, retrieval: can the system find your page for the question? That is classic indexing, covered in our indexing diagnosis. Second, selection: among the retrieved pages, which one contains a sentence the model can lift with confidence? That is where writing and evidence decide.

What does the research say gets pages cited by AI?

Three findings hold up, each from a study you can check yourself. None of them is a ranking formula; together they show where the levers are.

Sources, quotes and statistics raise visibility

The GEO paper by researchers at Princeton and IIT Delhi tested nine ways of rewriting a page on GEO-bench, a benchmark of 10,000 queries, and measured how much of the generated answer came from that page. Its abstract reports that GEO can boost visibility by up to 40%. In the paper itself the three best methods are Cite Sources, Quotation Addition and Statistics Addition, which gave a 30–40% relative improvement on its word-count metric and 15–30% on its impression metric. Keyword stuffing, the old SEO reflex, did not perform well.

A top-10 ranking helps, but no longer decides

Ahrefs analysed 863,000 keyword SERPs and about 4 million AI Overview URLs and found that 37.9% of cited URLs also appeared in the first 10 results. An earlier Ahrefs study had put the overlap near 76%; Ahrefs says its parsing changed, so the two numbers are not directly comparable. The practical lesson: pages beyond page one do get cited, if they answer the sub-question precisely.

Mentions of your brand matter more than links

Across 75,000 brands, Ahrefs found that branded web mentions had the strongest correlation (0.664) with being mentioned in AI Overviews, ahead of branded anchors (0.527), branded search volume (0.392) and Domain Rating (0.326). It is a correlation, not proof of cause, but it points the same way as the GEO paper: being talked about and quoted elsewhere is part of the signal.

ChatGPT leans on Bing’s results

Seer Interactive compared ChatGPT search citations with Bing and Google results for the same questions: 87% matched Bing’s top results, 56% Google’s. The authors call it a limited sample (about 100 queries, 500+ citations, early 2025), so treat it as a direction, not a law. The direction is clear enough: if Bing can’t index you, you are missing from a large share of ChatGPT answers. Our Bing Webmaster Tools guide covers the setup.

Which crawler feeds which AI answer?

Every assistant separates the crawler that powers its search answers from the one that collects training data. You can allow the first and block the second; blocking the search crawler removes you from that assistant’s citations.

OpenAI puts the consequence in one sentence: sites that opt out of OAI-SearchBot will not be shown in ChatGPT search answers, though they can still appear as navigational links. Note that the user-request fetchers (ChatGPT-User, Perplexity-User) may not follow robots.txt rules, according to their own documentation. The full robots.txt setup is in AI crawlers and robots.txt, and our Googlebot guide explains what Google-Extended does and doesn’t control.

How do you get cited by AI search, step by step?

1. Let the search crawlers in. Allow Googlebot, Bingbot, OAI-SearchBot, Claude-SearchBot and PerplexityBot in robots.txt and in your firewall. Decide separately about training crawlers.
2. Get indexed in Google and Bing. No index, no citation. Fix crawled or discovered — not indexed first, and push new URLs to Bing with IndexNow.
3. Answer in the first two sentences. Put the question in the heading and the answer directly under it, complete enough to stand on its own when quoted.
4. Add a number and a source to every key claim. This is the Statistics Addition and Cite Sources effect from the GEO paper. Link the primary source, not a summary of it.
5. Quote a named expert or document. Quotation Addition was the other top method. A short, attributed quote from official documentation counts.
6. Structure for extraction. One topic per section, short paragraphs, comparison tables, a FAQ in plain question form. Keep the important text in the HTML, not behind scripts or tabs that need clicks.
7. Keep snippets allowed. Google uses nosnippet, data-nosnippet, max-snippet and noindex to control what AI features may show. A restrictive max-snippet limits what can be quoted.
8. Get mentioned elsewhere. Original data, a free tool or a clear opinion earns the brand mentions that correlate with AI visibility. Answer engine and generative engine optimization are mostly this; see AEO vs GEO.
9. Keep it current. Put a visible update date on the page and change it only when the content changes.

How do you measure whether AI cites you?

• Bing Webmaster Tools → AI Performance. Since the public preview of February 2026 it counts citations in Microsoft Copilot, AI summaries in Bing and select partner integrations: total citations, cited pages, grounding queries and trends. In October 2026 Microsoft added Topics Portfolio, which groups those grounding queries into themes.
• Google Search Console. There is no separate AI report filter: Google says AI Overview and AI Mode traffic is counted in the Performance report within the “Web” search type. What Search Console does show about AI features is explained in the Generative AI report explained.
• Referral traffic. In your analytics, watch visits from chatgpt.com, perplexity.ai, claude.ai and copilot.microsoft.com.
• Manual spot checks. Ask your target questions in each assistant once a month and note who is cited. Keep the prompts identical so the trend means something.

What doesn’t get you cited?

• An llms.txt file. Google says you don’t need new machine-readable files, AI text files or markup to appear in its AI features. It is optional housekeeping at best; details in what llms.txt does and doesn’t do.
• Keyword stuffing. It was one of the methods that did not help in the GEO tests.
• Special “AI schema”. Google lists no special markup for AI Overviews. Normal structured data still helps Google understand the page; it is not a citation switch.
• Blocking every AI crawler and still expecting citations. You can’t have both; choose per crawler.

FAQ

A page stays out of Google for one of three reasons: Google has not found it, found it but has not crawled it, or crawled it and decided not to keep it — work through allowed → reachable → worth keeping, in that order.

1. Is the page even allowed to be indexed?

Before anything else, rule out the settings that tell Google to stay away. These are the fastest, most common fixes:

• A stray noindex tag (meta robots or the X-Robots-Tag HTTP header). This alone blocks indexing entirely — check the page source and response headers.
• Blocked in robots.txt. Blocking crawl means Google can’t read the page to index it. Confirm the URL isn’t disallowed.
• The whole site is set to “Discourage search engines” (WordPress → Settings → Reading). Easy to leave on after launch.
• A canonical pointing elsewhere. If the page defers to another URL, Google indexes that one instead. We cover this in depth in Duplicate, Google chose different canonical than user.

2. Can Google find and reach the page?

If the page is allowed but still absent, the issue is discovery or crawling — not quality. Check:

• Internal links. Orphan pages (nothing links to them) are the classic cause. Link the page from strong, already-indexed pages.
• Sitemap. The URL should be in your XML sitemap, and the sitemap submitted in Search Console.
• Click depth and server health. Pages buried deep, or served slowly / with 5xx errors, get crawled last.

3. Is the page worth keeping?

If Google has crawled the page and still won’t index it, this is a value judgement. Be honest with yourself:

• Is it substantially unique, or a near-duplicate of something you (or others) already published?
• Does it answer a real query better than what currently ranks?
• Is it thin? Consolidate several weak pages into one strong one and redirect.

Persistent “crawled – not indexed” on a healthy site is usually Google’s read on content quality or domain authority — not a technical glitch. The fix is making the page genuinely worth a spot.

Read your exact Search Console status

Search Console tells you which stage you’re stuck at — match the label to the fix (full list in Google’s Page indexing documentation):

Brand-new site and everything’s slow? That’s often normal — see how long Google takes to index before you worry.

Other statuses you might see

• Excluded by ‘noindex’ tag — the page tells Google to stay out.
• Blocked by robots.txt / Indexed, though blocked — crawling rules gone wrong.
• Soft 404 — the page looks empty or missing to Google.
• Duplicate without user-selected canonical and Alternate page with proper canonical tag — duplicates and canonicals.
• Page with redirect — usually normal; Redirect error is not.
• Sitemap could not be read / Couldn’t fetch — sitemap problems.

Related guides: How to submit your website to Google · How to check if a page is indexed · Orphan pages

Seeing a 403 or 401 in the report? That means Googlebot was refused at the door — usually by a firewall or bot filter. See blocked due to access forbidden (403).

Seeing a duplicate or canonical status? Our guide to canonical tags and indexing explains every one of them.

FAQ

On an established, healthy site a new, internally linked page is usually indexed within a few hours to about a week; on a brand-new domain, expect days to several weeks — there is no fixed timer, only trust, links and quality.

The three stages — and where the time actually goes

“Indexing” isn’t one step. A page moves through three, and the wait can happen at any of them:

The third stage — index — is where Google decides whether the fetched page is worth keeping. A thin or duplicate page can be crawled quickly and still never make it in. That’s a quality decision, not a speed one.

Typical timeframes (what “normal” looks like)

• Established site, well-linked page: a few hours to ~1 week.
• Established site, weakly-linked or deep page: 1–4 weeks.
• Brand-new domain: days to several weeks for the first pages, faster once the site earns trust.
• Thin, duplicate, or low-value page: may never index, regardless of time. Fix the page, not the wait.

How to get indexed faster (the levers that actually move it)

• Link the new page internally from strong, already-indexed pages. This is the single most reliable accelerator — it pulls the URL into the crawl queue.
• Keep it in your sitemap and make sure the sitemap is submitted in Search Console.
• Use URL Inspection → Request indexing once, for a genuinely ready page. Once — not daily.
• Fix crawl waste. If Googlebot burns budget on parameter URLs, redirects, or 5xx errors, your good pages wait longer.
• For Bing, use IndexNow. It notifies Bing and other engines the moment you publish — see what IndexNow is. (Google doesn’t use it.)
• Make the page worth keeping. Unique, useful content indexes faster and stays indexed. Speed is downstream of quality.

When to actually worry

Worry when a well-linked, high-value page on an established site stays unindexed past 3–4 weeks. Before then, on a young domain especially, waiting is the correct move. Panic-submitting the same URL every day does nothing but raise your blood pressure. Check the status in Search Console: if it says Discovered, it’s a discovery/priority issue; if it says Crawled – currently not indexed, it’s a value issue.

Related guides: How to check if a page is indexed · What is Googlebot? · The Google Indexing API

Crawling comes first. Before Google can index a page it has to fetch it — see how often Google crawls a site, with real Crawl Stats from this site.

FAQ

What does “Duplicate, Google chose different canonical than user” mean? You marked this URL as the canonical, with a rel=”canonical” tag or by listing it in your sitemap, but Google decided a different URL is the better version and indexed that one instead — not a bug, a disagreement.

What does “canonical” actually mean?

When two or more URLs show Google near-identical content, Google groups them and picks one to index and rank. That chosen URL is the canonical. The others are treated as duplicates and quietly set aside — they still exist, they just don’t get their own spot in the index.

You can suggest a canonical (with a rel=”canonical” tag, your sitemap, internal links, redirects). But Google treats your tag as a hint, not a command. When Google’s own signals point elsewhere, it overrides you — and that’s exactly what these Search Console messages are telling you.

Google’s own Page indexing report documentation describes this status in one line: “This page is marked as canonical for a set of pages, but Google thinks another URL makes a better canonical.” Google then indexes the URL it chose, not yours.

Which canonical statuses will you see, and what does each one mean?

A close cousin, “Alternate page with proper canonical tag,” is usually fine: it means the page correctly points to another canonical, and Google agreed. That one you can normally ignore — it’s the system working as intended. Every canonical status side by side is in our guide to canonical tags and indexing.

How do you check if Google chose a different canonical?

Each “no” is a signal you can align. The goal is simple: make every signal point to the same, single URL.

Two limits, both from Google’s URL Inspection documentation: the live test cannot predict whether the tested version will be chosen as canonical, because Google decides that at indexing time, and the indexed values can be a few hours old. Our guide to checking if a page is indexed walks through the tool step by step.

How do you fix “Google chose different canonical than user”?

• Make your signals agree. Your sitemap, your rel=”canonical”, your internal links, and any redirects should all name the same URL. Mixed signals are the number-one cause of Google overriding you.
• Self-reference the canonical. Every important page should point its canonical tag at itself (unless it genuinely is a duplicate). A missing or wrong tag invites Google to guess.
• Kill accidental duplicates. http vs https, www vs non-www, trailing slashes, tracking parameters, and printer/AMP variants all create twins. Redirect the variants to one canonical version; those variants then show up as Page with redirect, which is expected.
• Strengthen the URL you want. If Google chose a different page because it has more links, add internal links to your preferred URL from strong, already-indexed pages. A preferred URL with no internal links at all is an orphan page; fix that first.
• Consolidate genuinely thin duplicates. If two pages really are the same, don’t fight it — merge them and 301-redirect the loser. One strong page beats two competing ones.
• Rule out the less obvious causes. Google’s canonicalization troubleshooting guide also lists language variants without hreflang, incorrect canonical elements (often from a CMS or plugin), misconfigured servers, hacked pages, syndicated content and copycat sites.

When is a different canonical completely normal?

Plenty of these entries are harmless. Paginated pages, filtered/faceted URLs, and syndicated content should point to a canonical elsewhere — and “Alternate page with proper canonical tag” confirms that’s working. Only act when a URL you genuinely want indexed is being passed over for another.

FAQ

“Discovered – currently not indexed” means Google found your URL but never fetched it; “Crawled – currently not indexed” means Google fetched the page and declined to keep it. That one difference — fetched or not — decides which fix will work.

The one difference that changes your fix

Both statuses live in the same Google Search Console report, and people treat them as the same problem. They aren’t. The wall between them is a single question: did Googlebot actually fetch the page?

Get this right and everything downstream gets simpler. Chasing “crawl budget” when your real problem is thin content wastes weeks — a point practitioners on r/bigseo make repeatedly about pages “stuck for 8+ weeks” that no amount of re-submitting fixes.

Which one do you have? A 30-second decision

Answer the three questions in your column. Each “no” is a lever you can pull — in the order they’re listed.

Fixes that actually work (and the order to try them)

If it’s Discovered — make the page easy and worth reaching

• Add internal links from strong, already-indexed pages. This is the single most-cited fix in practitioner threads: a link from a page Google already trusts pulls the stuck URL into the crawl queue faster than re-submitting ever will.
• Flatten the click depth. Pages buried 5+ clicks deep read as low-priority. Surface them from hubs and category pages.
• Check your server logs for crawl waste. If Googlebot is burning its budget on parameter URLs, faceted filters, or 5xx errors, your good pages starve. Block the junk in robots and fix the errors.

If it’s Crawled — give Google a reason to keep it

• Raise the page’s unique value. Add first-hand detail, data, or a clearer answer than the current top results. “Crawled – not indexed” is often Google saying we already have this.
• Consolidate thin or near-duplicate pages. Ten shallow pages that each get rejected are worse than one strong page. Merge and redirect.
• Fix canonical confusion. If two URLs look the same to Google, it keeps one and drops the other. Make your canonicals explicit. See our full canonical guide.
• Then — and only then — request indexing. Use URL Inspection sparingly once the page genuinely deserves it. Re-submitting a weak page changes nothing.

The Indexing-API “trick” — why it’s tempting, and why it can backfire

You’ll find guides (and Reddit threads) showing how to force pages in via Google’s Indexing API — a step-by-step with a service account and a batch of URLs. It can work in the short term. But two things are true and usually left out:

• Google’s own docs say the Indexing API is only for JobPosting and BroadcastEvent pages. Using it for ordinary posts is off-label.
• Google has added an explicit spam warning to that documentation. Abuse can cost you far more than the pages were worth.

When “not indexed” is completely normal

Not every unindexed URL is a problem. New sites and new pages sit in these statuses for days to a few weeks while Google gauges the site. If your page is a week old on a young domain, the honest answer is: keep publishing, keep earning links, and wait. Panic-submitting daily does nothing but make you anxious.

Related guides: Orphan pages · What is Googlebot? · Why not to misuse the Indexing API

What do practitioners report actually worked?

We put the Reddit threads behind this guide through a structured source review (via NotebookLM, September 2026). Across r/SEO, r/bigseo, r/TechSEO and r/localseo, five fixes come back again and again as the ones that actually moved stuck pages:

• Internal links from strong pages. “I had this issue until I added more internal linking.” (r/SEO)
• A slight slug change, which makes Google treat the URL as new: “The only way I have to fix them is a slight slug edit.” (r/bigseo)
• A few backlinks to the stuck page: “Just sent a couple of backlinks to the page and it was fixed.” (r/SEO)
• Fixing page speed. One site owner with 2–3 minute load times: “I added a lazy load images code… about 3 days later, all the pages were indexed again.” (r/SEO)
• The Indexing API — one r/TechSEO test reported 94% indexed within 48 hours versus 8.4% via the sitemap. It works short-term, but it’s outside Google’s rules for normal pages; we explain the risk in our Indexing API guide.

The scale varies wildly: threads describe 3 stuck pages on a local-business site, 29 on a programmatic site after 8+ weeks, and 8,000 flagged on a news site. And the frustration with generic advice is real — “Anyone solved this beyond ‘earn backlinks and wait’?” (r/bigseo). That’s exactly why the split above matters: first determine which status you have, then apply the fix that matches it.

Is it crawl budget? On sites under about 10,000 pages, almost never. Google’s own thresholds are in crawl frequency and crawl budget explained.

Timing matters. A sudden jump in this status right after a Google update can be algorithmic — see our coverage of the September 2026 spam update.

FAQ