The web has hit a new milestone, as Google announces that its indexing system has found a trillion unique URLs.
In its blog, the web search giant said that its search engineers "stopped in awe" when they realised how big the web had become, after the index hit the trillion mark that's 1,000,000,000,000. The web is growing by several billion each and every day, Google added.
The first index by Google in 1998 found 26 million pages; the billion page mark was passed in 2000. Back in the day, Google batch processed everything, it said. Now, it's done constantly, with the entire set several times a day.
"This graph of one trillion URLs is similar to a map made up of one trillion intersections. So multiple times every day, we do the computational equivalent of fully exploring every intersection of every road in the United States," Google software engineers Jesse Alpert and Nissan Hajaj wrote in the blog.
Google said it's found more than one trillion pages, but that many are simply auto-generated copies. Indeed, the search giant doesn't even bother indexing all trillion unique pages, as some are too similar to each other, it said.
"So how many unique pages does the web really contain? We don't know; we don't have time to look at them all," they wrote, adding that "the size of the web really depends on your definition of what's a useful page, and there is no exact answer."
Google's trillion page announcement comes as a rival called Cuil, founded by former Google employees, hits the web, with claims that it can index more of the web, faster and more cheaply than Google. At its launch, it said it had indexed 120 billion pages.
-
Bigger salaries, more burnout: Is the CISO role in crisis?In-depth CISOs are more stressed than ever before – but why is this and what can be done?
-
Cheap cyber crime kits can be bought on the dark web for less than $25News Research from NordVPN shows phishing kits are now widely available on the dark web and via messaging apps like Telegram, and are often selling for less than $25.
-
Researchers outline real-time search engine plansNews Glasgow university computer scientists want to give city dwellers access to a real-time information source by 2014.
-
Google promises 'fresher' searchNews The internet giant looks to make its searches more timely.
-
Yahoo takes a bite out of Google's search leadNews The latest comScore's latest search engine figures for the US sees Yahoo rise and Google fall as third-placed Bing prepares to take over Yahoo's North American search duties.
-
Bing lockdown for Windows Phone 7 handsets?News Integration with the Tell Me voice-control feature means Windows Phone 7 devices will be Bing-only by default, a situation even OEMs won't be able to change.
-
Caffeine peps up Google's search engineNews Continuous indexing means results are fresher and faster, says Google, taking the fight to Bing but moving the goalposts for SEO professionals.
-
Google adds page load times to search rankings formulaNews Speed is added to the mysterious Google page rankings melting pot as search giant tries to encourage webmasters to take a closer look at page load times.
-
Wolfram Alpha drops app price from $50 to $2News Wolfram Alpha has announced a dramatic price drop and refunds to attract more users.
-
Google sees Hong Kong traffic jumpNews Google Hong Kong's market share has jumped to three per cent of global market share - if it falls back, it may be a sign China has cut access.
