SSD-0006 · Status: contested · Version 1.0 · Reviewed 12 November 2026 · Wikidata: no item
Definition
Crawl budget is the number of URLs a search engine is willing and able to fetch from one site in a given period — a function of how much the site deserves crawling and how much capacity the crawler has to spend.
檢索預算是指搜尋引擎在特定期間內願意且能夠從單一網站抓取的網址數量——取決於該網站值得檢索的程度，以及檢索器本身有多少容量可以花。
In practice
The term is used mainly to explain one symptom: pages that are discovered but not indexed, or indexed slowly. For most sites under a few thousand URLs, crawl budget is not the constraint — the constraint is that the pages are not worth crawling. It becomes real at scale. Faceted navigation, filter parameters, session IDs and near-duplicate archives can generate millions of URLs that consume crawl capacity while never earning a visit. The diagnostic is a server log, not a tool score: how often is each URL shape fetched, and what comes of it.
這個詞主要用來解釋一個症狀：頁面已被發現但未被索引，或索引得很慢。對大多數幾千個網址以下的網站，檢索預算不是瓶頸——瓶頸是那些頁面本身不值得檢索。到大規模才真正成立。多面向導覽、篩選參數、session ID 與近乎重複的存檔頁，可以產生數百萬個消耗檢索容量卻從不帶來訪客的網址。診斷工具是伺服器日誌，不是工具分數：每種網址型態被抓取多少次，又有什麼結果。
What it is not
- Not a published quota. No search engine states a site’s allowance, and no tool can measure it.
- Not the same as a crawl rate limit. A rate limit is the site asking a crawler to slow down; crawl budget is the crawler deciding how much to spend.
- Not a ranking factor. It affects discovery speed, which affects visibility timing, not position.
Status
Contested. Google has publicly played the term down, describing crawl capacity as site-specific and rarely the limiting factor for ordinary sites; practitioners use it as shorthand for a real phenomenon at scale. Both are true at different sizes. Treat any “crawl budget score” produced by a tool as a vendor construct, not a measurement.
Identity
- Wikidata: no item (checked 12 November 2026)
- English aliases: crawl capacity; crawl demand
- 中文正名：檢索預算（香港、台灣）；中國大陸作「抓取预算」
- Primary official source: Google: managing crawl budget for large sites
Related terms
Sources
- Google Search Central, “Managing crawl budget for large sites”, developers.google.com/search/docs/crawling-indexing/large-site-managing-crawl-budget (accessed 12 November 2026).
- Wikidata entity search for “crawl budget”, no matching item (checked 12 November 2026).