CS262B Advanced Topics in Computer Systems
Spring 2009

David E. Culler

A new OS architecture for heterogeneous multicore systems analysis

Provide a short summary of the paper:
This paper advocates for a redesign of operating systems to reflect the
distributed nature of the underlying hardware. Future generations of
hardware will offer larger numbers of cores that are also heterogeneous. At
the same time, the interconnect between these cores will be subject to many
of the same problems currently experienced in networks. Furthermore, the
larger number of cores will make cache coherency much more complicated and
make it more difficult to provide the shared-memory abstraction. So, the
authors propose that new operating systems be composed of multiple kernels,
one for each core, and that those kernels communicate strictly through
well-defined message-passing interfaces. Then, operating systems will look
very similar to distributed systems and will be able to leverage all of the
advances in that field.
What is the strength of this paper:
The authors of this paper provide an actual operating system implementation
that they benchmark on a variety of machines to substantiate their claims.
As such, they are able to demonstrate very good scaling characteristics.

What is the weakness of this paper:
 The authors are unable to ensure that the performance gains witnessed as
compared to Linux and Windows are due specifically to the multi-kernel model
advocated for in this paper. It is possible that some of these differences
are due to implementation artifacts or other factors.
Your qualifications to review this paper:
I know the material, but am not an expert.


Writing quality:
Good.
Relevance to SOSP:
People would read it before conference and attend session.
Experimental Methodology:
Good.
Novelty of Paper:
This is a new contribution to an established area.
Overall paper merit:
Strong accept.
Additional comments:
Would it have been possible to show the incremental improvements of each
aspect advocated for in the paper? For example, could blocking and
non-blocking operations be compared for a workload in AnonOS? Similarly,
could shared memory and message-passing costs be compared within AnonOS?
This would ensure that all of the gains witnessed are due specifically to
the functions advocated for in this paper.

Short summary: The authors argue that an operating system should be treated like a distributed message passing system where shared state is made explict rather then implicit.  They contend that this would allow a machine to consist of very hetrogeneous cores running different systems software and yet still communicating, and scaling to very large numbers of these cores.  The design has a preliminary implementation which reveals no significant performance cost on current hardware for this approach.

Paper strengths: the design is of current interest, and using current shared-memory machines to implement a message passing system is easy and convincing because the architectures are fundamentally message based, under the hood.  There are several strong arguments made in the paper for this architecture such as dealing with hetrogeneity, especially citing network and graphics cards as "too smart" to be accesed as dumb peripherals like current techniques require as well as adding the ability to batch updates for more efficient bus utilization-- the typical message system arguments. 

Paper weaknesses: The exact design of their operating system seems strangely vague to me considering they have actually implemented it.  Processes are run using something like scheduler activations, but how memory sharing is coordinated between these processes (they say it's possible) doesn't seem clear; they really have two designs going on, one which would be on an ideal architecture which is natively message based, and one which is actually implemented on shared hardware.  How exactly do mechanism like vm work in the implementation, and does this differ from how it would on the shared architecture?  Where does the networking stack go?  Some of this is burried in there, but could be much better organized.  Furthermore, what are the implications of the split phase nature?  I think a few examples of how you would implement kernel subsystems would really help.  In general, what hardware architecture are they proposing?  It's worth noting that message systems (microkernels) did not originally take off more or less because the constant was too big, and this appears in their charts, for a single core.  They're arguing that this has changed now that we have lots of cores and what we care about is further out on that graph.  The fundamental arguement of this paper is that systems should be based on message passing.  Good arguement.  However, by introducing an actual system, they add a huge number of questions which can't really be answered in the same paper as a position paper arguing for messages.  I'm not sure they quite manage to finesse these two tasks.

Qualifications: 3?

Writing quality: excellent.

Relevence to SOSP: 4 or 5?

Experimental methodology: the preliminary implementation is an important step towards evaluating the proposed architectual change, and seems sufficient for a first step.  This feels like a very reasonable thing to do and furthermore is well motivated, so the burden of proof may be somewhat lower.  The mix of macro- and micro-benchmarks serve to illuminate the strengths while arguing that the architecture has no inherrent inefficiencies.  It would have been nice to see a carefully engineered implementation of one of the numerical tasks achieve an optimial speedup.  The loopback tests and 9(c) especially leave me somewhat curious since it's not clear that, in the case of 9(c) the performance of the systems "do not differ significantly."

Novelty: they cite a lot of work, although interestingly I wish there was actually more discussion about the difference between this system and microkernels; they do mention exokernels and two true microkernel references, but I would like some more discussion about this.  [I think] the most novel element is advocating exposing message passing among hetrogeneous hardware, although there certainly are things like this in the supercomputer world-- maybe this is an indication that this is a good idea, since those systems are definitly scalable.

Overall Paper merit: Accept

I agree with many of their arguements, and think that the implementation does a reasonable job of making the point that there is not inherent bottleneck.  However, the real coup would be demonstrating a system where some significant benefit is attained or else integrates hetrogeneity much better then is possible today.  Their mention of an ARM port makes me think they agree with this point.  There are several very good papers in all this work, and the question is how to divide it.  I'm not sure this piece exactly hits the nail on the head but it's a nice piece of work, with a lot of nice engineering techniques.

Summary: This paper focuses on the problem that hardware is changing and diversifying faster than operating system software. The internal organization of a general-purpose computer increasingly resembles a network rather than a single system; therefore it is no longer useful to tune a general purpose OS design for a particular hardware model. The approach that this paper takes is to model a new OS structure, the multikernel, that treats the machine as a network of independent cores, assumes no inter-core sharing at the lowest level, and moves traditional OS functionality to a distributed system of processes that communicate via message-passing. The authors provide the main design principles of replication, message passing and split-phase invocation and implement the multikernel model (AnonOS). They show that AnonOS is scalable and adapts to a diverse set of hardware characteristics, while providing competitive performance.

What is the strength of the paper? (1-3 sentences) The main contribution of this paper is to treat the machine as a network of independent cores and then to make use of previous ideas in distributed systems. I believe that this core idea has a lot of potential for expansion. Furthermore, this paper is very well structured with a clear problem statement, the identification of the design principles for their model and a detailed justification for these choices, and finally an implementation and evaluation of the model.

What is the weakness of the paper? (1-3 sentences) One weakness of the paper is not to present the negative aspects of their work and to state some issues that they noticed while implementing AnonOS.

Your qualifications to review this paper: 2
Writing quality: 5
Comment: The writing was clear and I liked how the authors motivated each part and the design decision of the paper.
Relevance to SOSP? 5
Comment: I think it is a very interesting paper and tackles a problem relevant to the community.
Experimental methodology: 4
Comment: There is a nice parallel between section 5 and 3.4 where the authors evaluate all the goals stated previously, but in some cases the presentation could have been a little clearer.
Novelty of paper: 5
Comment: Their idea of treating the machine as a network of independent cores with no directly shared memory is, to the best of my knowledge, very novel.
Overall paper merit: 5
Provide additional detailed comments to the author: I have written the comments with each question instead.

This paper proposes the re-evaluation of the design of multicore operating systems as a distributed networked system.  By viewing each core as a separate processing unit with its own state and using message-passing and various distributed-system protocols to manage consensus and sharing, you can generalize the operating system for the ever changing underlying hardware.  A general message-passing interface and split-phase programming can allow for pipelining and other optimizations that decouple the techniques and mechanisms from the policies for dealing with sharing across core.  Furthermore, many of the contributions and work in the distributed system theoretical and systems literature can be pulled into future OS design for multicore systems.

Strengths:

The authors bring up many convincing arguments and motivate the problem quite well.  Furthermore, the AnonOS implementation of the multikernel design is a nice demonstration that this is more than just an academic discussion.  The fact that it was able to perform comparably to current systems is encouraging.

Weaknesses:

It's not clear to me how much of the performance overhead seen in the evaluation is inherent to their implementation of the design and how much is the actual design of a multikernel system.  To the authors' benefit, however, they do state that the evaluation is not thorough and that instead what should be focused on is the fact that the performance numbers are comparable which motivates the design choices.

 
Your qualifications to review this paper: 3

Writing Quality: 5

Relevance to SOSP?: 5

Experimental Methodology: 4

Novelty of paper: 4

Overall paper merit: 5

Provide additional detailed comments to the author:

This is a nice piece of work that focuses the community on thinking about new designs for multicore system and I think that going about it like this paper has will allow operating system designers to more easily keep up with the rate of change of hardware.  That said my only concern is that if this does become a viable design point, that more distributed system problems will be introduced at the cost of performance that could/would have been solved more easily with current techniques.  In other words, if you choose to introduce the benefits of distributed system design into OS design you also introduce the inherent set of problems that that community is also trying to tackle.  Small problems in more distributed higher-latency network settings might be amplified at the scale of a multicore OS.

Additional comments to PC:

I think this is a fresh look at multicore OS design.  It's important, i think, for the community to consider this work in moving forward and keeping up with the hardware trends and to move away from incremental improvements and tweaks made to new systems based on specific design decisions made for the older versions.

Provide a short summary of the paper:

This paper starts from the premise that hardware is evolving too rapidly for OS software to keep up with it.  Furthermore, hardware is becoming less uniform over time and the current strategy of treating any unusual processing units as off chip devices doesn't scale well.  The solution proposed by the paper is to treat the processor more like a distributed system and have the OS be programmed like a distributed system.  Each core is its own processing node and everything communicates via message passing.  This allows an OS to be easily adapted to different architectures by simply optimizing the message passing code.  The rest of the OS can remain unchanged.  Also described is an implementation of an OS called AnonOS using the ideas presented in the paper.  It is shown that AnonOS scales better than existing OSes on some workloads, and as well on others even though AnonOS has not been optimized much at all.

What is the strength of the paper? (1-3 sentences) 

The paper makes a strong argument for why message passing is a preferable paradigm for programming an OS in.  Furthermore, they show that existing OSes employ techniques that simply don't scale well in multi-core NUMA environments.

What is the weakness of the paper? (1-3 sentences) 

The paper uses one example to argue why existing OSes will not be able to adapt to new architectures well.  This is unconvincing for two reasons.  Firstly, perhaps it was simply bad design in Windows that made it so much work to remove the dispatcher lock, and secondly, implementing a whole new OS is a huge amount of work too.

Your qualifications to review this paper

• I know the material, but am not an expert

Writing Quality

• Good

Relevance to SOSP?

• People would read it before conference and attend session

Experimental Methodology:
Average

Novelty of paper:
This is a new contribution to an established area

Overall paper merit:
Strong accept - This is of interest to Sensys, and a novel or new contribution to existing area with good/average methodology, or an incremental contribution paper that has excellent methodology.
      
Provide additional detailed comments to the author:
The presentation is good and the ideas are certainly interesting.  I would like to see some absolute performance comparisons to existing systems.

     
Additional comments to PC:
Similar work has been done before (Tornado/K42), but this is still an interesting paper and worth reading.