Declarative Distributed Debugging (D3)
Byung-Gon Chun, Kuang Chen, Gunho Lee, Randy H. Katz and Scott Shenker
Computer applications are increasingly distributed, running over many nodes. Debugging such distributed systems is hard. Typically, applications create sizable ad-hoc logs on each node. To debug a problem, the system or user must aggregate logs centrally and filter logs to remove a large amount of irrelevant data. Such a task is like finding a needle in a haystack. It requires a great amount of human labor and network bandwidth. This poses a challenge to the scalability of distributed systems.
To address this problem, we propose D3--a framework for more formal and efficient distributed debugging. We approach the challenge of distributed debugging by specifying queries declaratively, and distributing log-query processing to the nodes. Building on the P2 distributed query engine  and the X-Trace  tool for tracing, we aim to simplify distributed debugging and reduce network load.