Tech Talk: One Log to Rule Them All – How to Keep State Straight in Distributed Systems

Hosted by Jane Street and presented by Doug Patti

At Jane Street, we rely heavily on inter-process communication and real-time message passing to drive many of our most critical systems, including those that ingest market data and implement trading strategies. As we scale up the number of components, however, it becomes increasingly difficult to reason about the consistency and correctness of the overall system.

In this talk, we explore how transaction logs, state machine replication, and control flow inversion have allowed us to structure our applications in a way that lets us reliably deal with data races and process failures. We introduce Aria, a low-latency framework built on top of these ideas that has helped Jane Street scale our trading infrastructure while making it more performant, testable, and robust.