[Air-l] Re: History of Google

[Jay Hauben]

Hi,

An important piece of the history of Google is contained in the paper 
by Larry Page and Sergey Brinn, The Anatomy of a Large-Scale 
Hypertextual Web Search Engine, at

http://www-db.stanford.edu/~backrub/google.html

In that paper Page and Brinn explain the original motivation for the 
development of the Google search engine as an alternative to the 
dominant commercialization of search engine research . They wrote 
there:

"1.3.2 Academic Search Engine Research
Aside from tremendous growth, the Web has also become increasingly 
commercial over time. In 1993, 1.5% of web servers were on .com 
domains. This number grew to over 60% in 1997. At the same time, 
search engines have migrated from the academic domain to the 
commercial. Up until now most search engine development has gone on at 
companies with little publication of technical details. This causes 
search engine technology to remain largely a black art and to be 
advertising oriented (see Appendix A). With Google, we have a strong 
goal to push more development and understanding into the academic 
realm.

Another important design goal was to build systems that reasonable 
numbers of people can actually use. Usage was important to us because 
we think some of the most interesting research will involve leveraging 
the vast amount of usage data that is available from modern web 
systems. For example, there are many tens of millions of searches 
performed every day. However, it is very difficult to get this data, 
mainly because it is considered commercially valuable.

Our final design goal was to build an architecture that can support 
novel research activities on large-scale web data. To support novel 
research uses, Google stores all of the actual documents it crawls in 
compressed form. One of our main goals in designing Google was to set 
up an environment where other researchers can come in quickly, process 
large chunks of the web, and produce interesting results that would 
have been very difficult to produce otherwise. In the short time the 
system has been up, there have already been several papers using 
databases generated by Google, and many others are underway. Another 
goal we have is to set up a Spacelab-like environment where 
researchers or even students can propose and do interesting 
experiments on our large-scale web data."

----------------------------

This original academic purpose for Google was turned into a commercial 
purpose under the urging of the NSF.

Hope this bit of history helps.

Take care.

Jay