Organising information – how Google Search works https://www.google.com/search/howsearchworks/crawling-indexing/ Jump to Content Overview Our approach Accessing info Info quality Ads News Families and children Helping creators How Search works Organising information Ranking results Rigorous testing Detecting spam Features Our history How Google Search organises information When you search, Google looks through hundreds of billions of web pages and other digital content organised in our Search index to find helpful information – more information than all of the libraries of the world. Finding information Sorting content Processing changes Building indexes Finding information by crawling Most of our Search index is built through the work of software known as crawlers. These automatically visit publicly accessible web pages and follow links on those pages, much like you would if you were browsing content on the web. They go from page to page and organise information about what they find on these pages and other publicly accessible content in Google's Search index. Sorting and organising content When crawlers find a web page, our systems render the content of the page, just as a browser does. We take note of key signals – from keywords to website freshness – and we keep track of it all in the Search index. The Google Search index covers hundreds of billions of web pages and is well over 100,000,000 gigabytes in size. It's like the index in the back of a book – with an entry for every word seen on every web page that we index. Google never accepts payment to crawl a site more frequently – we provide the same tools to all websites to ensure the best possible results for our users. Constantly processing new info Because the web and other content is constantly changing, our crawling processes are always running to keep up. They learn how often content they've seen before seems to change, and revisit as needed. They also discover new content as new links to those pages or information appear. Google also provides a free toolset called Search Console that creators can use to help us better crawl their content. Creators can also make use of established standards like sitemaps or robots.txt . Google never accepts payment to crawl a site more frequently – we provide the same tools to all websites to ensure the best possible results for our users. Indexing more than web pages Our Search index covers more than just what's on the web, because helpful information can be located in other sources. In fact, we have multiple indexes of different types of information, which is gathered through crawling, through partnerships, through data feeds being sent to us and through our own encyclopaedia of facts, the Knowledge Graph . These many indexes mean that you can search within millions of books from major libraries, find travel times from your local public transport operator or find data from public sources like the World Bank . Discover more rigorous testing Always improving your experience Learn more ranking results Instantly matching your search Learn more detecting spam Keeping you safe on Search Learn more The Keyword Nine ways that we use AI Learn more Footer links Our approach Maximise access to information Sell ads, nothing more Help creators succeed online How Search works Organising information Our approach Rigorous testing Detecting spam Features For children and family safety For evaluating information Resources Search Essentials Your data in Search Rater guidelines overview When (and why) we remove content from search results More about us Blog Safety Centre Economic impact Sustainability Public Policy Transparency Centre About Google Google products Privacy Terms Help British English Australian English Dansk Deutsch English English (India) Español Español latinoamericano Estonian Français Français canadien Gaeilge Hrvatski Indonesia Italiano Latviešu Lietuvių Magyar Maltese Nederlands Norsk Polski Portuguese Português (Brasil) Pусский Română Slovenčina Slovenščina Suomi Svenska Tiếng Việt Türkiye Yкраїнська čeština Ελληνικά български עברית العربية हिन्दी ไทย 中文 (台灣) 日本語 한국어