Google file system, large scale architectures, Smalltalk Open Croquet

One core piece of infrastructure behind the Google search engine is the Google file system. If you have not already read this paper (PDF) then I suggest that you do so.

Starting with the requirement to be able to efficiently merge work queue data (e.g., spidered links) lead to the design of a file system that supported very inexpensive file append operations.

To me, Google represents at this time the best technology for mega-scale centralized processing. At the other end of the spectrum of large scale design is the Open Croquet project: a design for supporting very large scale peer to peer collaborative environments. The Open Croquet white papers give a great overview to this system. Open Croquet (due to be re-released within a month) combines a scalable immersive working environment with 3D virtual reality (VR) technologies. Open Croquet is vaguely similar to Sun's 3D Looking Glass desktop in appearance, but is really something very different.

Open Croquet covers two of my interests: large scale architecture and VR. (I was a principle in the SAIC VR Systems Division, and was the lead on a VR project for Disney when I worked at Angel Studios). I think that Alan Kay is correct: in computer science, we are just beginning. The real fun is yet to be experienced!