Term · Search & content
Indexing
Indexing is the process by which a search engine analyses a page and stores it in its index — the database from which search results are drawn. A page that is not indexed cannot appear in search results, however good it is.
[ 01 ]
Explained
From discovery to results
- Discovery — the search engine learns a URL exists, through links or sitemaps.
- Crawling — its crawler fetches the page.
- Rendering — it processes the page, including JavaScript where it does so.
- Indexing — it analyses content, decides whether to store the page and how to understand it.
- Ranking — for each query, indexed pages are ordered by relevance and quality.
Why pages are not indexed
- Blocked by robots.txt or marked
noindex. - Duplicate of another page, with the other chosen as canonical.
- Low quality or thin content.
- Not discovered: no internal links, not in the sitemap.
- Rendering problems: content missing from the HTML crawlers see.
- Technical errors: server errors, redirects, soft 404s.
- For large sites, limited crawling capacity. See crawl budget.
How to check
- Google Search Console's page indexing report and URL inspection tool.
site:searches give a rough indication, not an exact count.
Controlling indexing
- To exclude a page:
noindexmeta tag or header, or authentication. - To encourage indexing: internal links, sitemap inclusion, unique valuable content, correct canonical tags.
[ 02 ]
FAQ
Frequently asked questions
How long does indexing take?
From hours to weeks, depending on the site's authority, crawl frequency and the page's discoverability.
Should every page be indexed?
No. Search results, filters, thin tag pages and private areas are better kept out.