Home / Guides / Crawl budget
Crawl budget
Crawl budget is the set of URLs that Google can and wants to crawl on a site. It depends on how much crawling the server can handle and on how much Google wants to crawl, which reflects the site's size, update rate, page quality and popularity. Each hostname has its own.
Updated 9 Nov 2026
In short
Most sites do not need to worry. Google's guide is aimed at sites with around a million or more unique pages that change weekly, sites with around ten thousand or more pages that change daily, and sites with many URLs reported as 'Discovered, currently not indexed'. Google calls these rough estimates, not exact thresholds. A US or Canadian local business site with a few hundred pages is well below them.