Skip to content
PromotEdge International

Home Guides Crawl budget

Crawl budget

Crawl budget is the set of URLs that Google can and wants to crawl on a site. It depends on how much crawling the server can handle and on how much Google wants to crawl, which reflects the site's size, update rate, page quality and popularity. Each hostname has its own.

Updated 9 Nov 2026

In short

Most sites do not need to worry. Google's guide is aimed at sites with around a million or more unique pages that change weekly, sites with around ten thousand or more pages that change daily, and sites with many URLs reported as 'Discovered, currently not indexed'. Google calls these rough estimates, not exact thresholds. A US or Canadian local business site with a few hundred pages is well below them.

What to do if it applies to you

Keep the sitemap current, remove or block low-value duplicate URLs, fix server errors and slow responses (which lower the crawl limit), and check the Page Indexing report in Search Console. Source: Google Search Central, crawl budget for large sites, read 9 November 2026.

Our technical audits cover crawling and indexing checks.

Tell us what is not working

A real person reads every message and replies.

Chat with us