The Saga Pattern: Distributed Transactions Across Microservices

A single order touches three services. Payments charges the card, inventory reserves the item, shipping schedules the delivery. Each owns its own database, and no transaction spans all three. So when shipping fails after payments has already captured the money, you are left with a charge, a reservation, and no delivery. Someone has to walk back the first two steps. That coordination problem, and how you talk through it, is the saga pattern and one of the sharpest system design interview questions.

No transaction spans three services

Place an order and three separate services each commit to their own database. There is no shared connection, no shared transaction, nothing that lets you roll all three back together.

Run the steps in order and the third one fails. Payments committed. Inventory committed. Both are done, in their own databases, and there is no ROLLBACK that reaches across a service boundary to a transaction that already finished. The money is gone, the stock is held, and the customer is waiting for a package that will never ship.

In code, the naive version is just three calls in a row. When the third throws, the first two have already committed and nothing undoes them:

// No coordination: three commits to three databases, nothing undoes a partial run.
void placeOrder(Order order) {
    payments.capture(order);    // committed to the payments DB
    inventory.reserve(order);   // committed to the inventory DB
    shipping.schedule(order);   // throws: out of capacity
    // money captured, stock reserved, no shipment, no rollback reaches the first two
}

The instinct is a distributed transaction across all three. Same trap as before: 2PC blocks on a coordinator, tanks availability, and couples services that were supposed to stay independent. The outbox pattern already ruled this out for one service and a broker. It is no better across three services. Don't.

A saga is a chain of local transactions

A saga breaks the distributed transaction into a sequence of local ones. Each step is an ordinary ACID transaction inside a single service. When a step commits, it triggers the next. When a step fails, the saga runs compensating transactions to undo the steps that already committed, in reverse order.

Each hop is durable on its own. No step waits on a lock held by another service, and nothing blocks the way a two-phase commit would.

Failure means compensation, not rollback

You cannot roll back a committed transaction in another service. You issue a new transaction that semantically undoes it. A capture is undone by a refund, a reservation by a release. These are compensating transactions: business-level inverses, not database rollbacks.

Every forward step needs a matching backward step, defined up front:

The compensation is visible to the customer. A refund is a real event, not a silent internal undo, so you design it deliberately rather than pretending the step never happened.

In code, that reverse unwinding is a stack you build as you go and pop on failure:

// Register each compensation only after its step commits, then unwind in reverse on failure.
void placeOrder(Order order) {
    Deque<Runnable> compensations = new ArrayDeque<>();
    try {
        payments.capture(order);
        compensations.push(() -> payments.refund(order));

        inventory.reserve(order);
        compensations.push(() -> inventory.release(order));

        shipping.schedule(order);
        compensations.push(() -> shipping.cancel(order));
    } catch (RuntimeException err) {
        compensations.forEach(Runnable::run); // LIFO: release stock, then refund
        throw err;
    }
}

This linear driver is the orchestration style covered below; choreography reaches the same end state through failure events instead. It leaves out two things on purpose, both further down: each step also writes its event to an outbox in the same local transaction, and each compensation must be idempotent because it can be retried.

Two ways to run a saga

The steps are clear. What is left is who drives them, and this is where the interview turns. There are two coordination styles, and picking between them is the real question.

Choreography: services react to events

No central brain. Each service subscribes to the events it cares about, does its local transaction, and emits the next event. Payments emits PaymentCaptured, inventory listens and reserves, then emits StockReserved, shipping listens and schedules. On failure a service emits a failure event and the upstream services compensate when they see it.

What it wins you:

• No coordinator to build, run, or scale.
• Services stay decoupled. Each one only knows the events it consumes and emits.
• Adding a step is often just subscribing to an existing event.

What it costs you:

• No single place tells you the whole flow, so debugging means tracing events across services.
• Event chains can grow cyclic, and a bad handler can set off a storm.
• Answering "where is order 9F2 right now" means reconstructing state from a log.

Orchestration: a coordinator drives the flow

One component, the orchestrator, holds the saga as a state machine. It calls each service, waits for the result, and decides the next move, including which compensations to run when a step fails. The services stay dumb; the sequence lives in one place.

What it wins you:

• The whole flow lives in one readable place, and you can query exactly where a saga is.
• Complex logic, branching, timeouts, and retries are easy to express.
• The orchestrator is straightforward to test in isolation.

What it costs you:

• Another service to build, deploy, and keep available.
• It can drift into a god service that hoards business logic the domain services should own.
• Every saga now depends on the coordinator being up.

Choreography vs orchestration

Start with choreography for a short, linear flow. Reach for an orchestrator when the number of steps, branches, or timeouts makes "where are we" a question no one on the team can answer quickly.

The tradeoffs that get you the offer

Naming the pattern is table stakes. These follow-ups are what separate a senior answer in a system design interview.

Sagas drop the I in ACID

A saga is atomic in the end, consistent, and durable, but it is not isolated. Between the second step and the last, other transactions can see the half-finished state: an order that is paid but not shipped, stock reserved for an order about to be cancelled. There is no lock spanning the saga. Say "ACD, not ACID" out loud, then name a countermeasure: a semantic lock (a PENDING status that other work checks and skips), commutative updates so step order stops mattering, or rereading the value right before you act on it.

Every step still needs the outbox

A saga step commits its local change and then has to publish the next event. That is the same dual-write problem the outbox pattern exists to solve: the commit succeeds, the publish fails, and the saga stalls with no one holding the two together. Write the event to an outbox table in the same local transaction as the step, and let a relay deliver it. Both choreography and orchestration need this for reliable handoffs.

Compensations must be idempotent, and they can fail

At-least-once delivery means a compensation can arrive twice, and refunding twice is a real bug, not a hypothetical. Key every compensation by orderId plus step so a repeat is a no-op. And a compensation can itself fail: the refund service is down exactly when you need it. Retry with backoff, and when the retries are exhausted, route to a dead-letter queue for a human. A compensation that can never fail is a fantasy, so design the escape hatch before the interviewer asks for it.

Some steps cannot be undone

You cannot un-send an email or un-call an external partner. Order the saga so every irreversible action comes last, after every compensable step has already succeeded. That boundary has a name, the pivot transaction: everything before it is compensable, everything after it is retriable but not undoable. Put the point of no return where you can actually defend it.

Reading the pattern is not the same as defending it on a whiteboard. Wire a saga into a real payment system design: choose choreography or orchestration, put an outbox on every step, and talk through what happens when a compensation fails.