Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

Very inspiring work and achievement. It is perhaps the equivalent of the application of quantum mechanics during the middle to late twentieth century. As such, perhaps "decentralized general computing" is more plausible than I thought.


It's a great paper, but I think you're hugely exaggerating its significance.

As the authors themselves point out, none of the fundamental building blocks of this system are particularly new. For example, the idea of partitioning a very large dataset into lots of independent slices, each of which is handled by its own Paxos group, is the same idea that forms the basis of Google's Megastore and Spanner, the former of which is more than a decade old.

Most of the interesting stuff in this paper is the discussion of the nuts-and-bolts of software engineering, such as testing, deployment and monitoring.


People find this stuff exciting because most of our industry works on systems that are at least ten years behind the state of the art. A good example is HDFS, an extremely bad likeness of GFS which itself wasn't great and died ten years ago. If you describe Colossus/D in detail many people in our industry will think it's really amazing, but of course that's more than ten years old now. Many people will choose HDFS for new systems in new designs, today. You can spend your whole career without getting so much as a whiff of the state of the art.


And what are the better modern alternatives to hdfs among distributed file systems?


I would say that a distributed filesystem is a solution looking for a problem in most cases. Amazon S3 or Google Cloud Storage address some use cases, and Google Cloud BigTable is a direct drop-in replacement compatible with the HBase API but having dramatically better performance and reliability. There are other use cases that have other alternatives, it all depends on what you plan to do with the data on the filesystem, how far you need to scale it, and whether you clients are in your own datacenters or in vendor clouds.


> And what are the better modern alternatives to hdfs among distributed file systems?

For open source solutions, BeeGFS.

If you want to pay and you are IBM fan, GPFS.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: