Electrical Engineering
      and Computer Sciences

Electrical Engineering and Computer Sciences


UC Berkeley


2010 Research Summary

Nexus - A Common Substrate for Cluster Computing

View Current Project Information

Benjamin Hindman, Andrew Konwinski, Matei Zaharia, Ali Ghodsi1, Anthony D. Joseph, Scott Shenker and Ion Stoica

Cluster computing has become mainstream, resulting in the rapid creation and adoption of diverse cluster computing frameworks. We believe that no single framework will be optimal for all applications, and that organizations will instead want to run *multiple* frameworks in the same cluster. Furthermore, to ease development of new frameworks, it is critical to identify common abstractions and modularize their architectures. To achieve these goals, we propose Nexus, a low-level substrate that provides isolation and efficient resource sharing across frameworks running on the same cluster, while giving each framework maximum control over the scheduling and execution of its jobs. Nexus fosters innovation in the cloud by letting organizations run new frameworks alongside existing ones and by letting framework developers focus on specific applications rather than building one-size-fits-all frameworks.

B. Hindman, A. Konwinski, M. Zaharia and I. Stoica, A Common Substrate for Cluster Computing, HotCloud 2009, June 2009. Available at http://www.cs.berkeley.edu/~matei/papers/2009/hotcloud_nexus.pdf.

1UC Berkeley