Lesson 15: Monolith vs Microservices

The question "should we split into microservices" usually gets answered by gut feeling, because the cost of splitting is hard to picture until you have actually paid it. By this lesson we already have: Lesson 4 showed that every hop adds latency, Lesson 9 showed what eventual consistency feels like, Lesson 10 showed how hard a distributed lock is, Lesson 11 showed why everything has to be idempotent. That is why this lesson sits at position 15, not position 3.

The lab measures that cost in numbers. Same four-step use case, same amount of work per step: running in one process gives p99 2.06 ms and 18,887 req/s; splitting it into HTTP calls gives p99 8.25 ms and 3,744 req/s — 4 times slower at the tail and 5 times less throughput. With each service failing 1% of the time, an 8-hop chain measures a success rate of 92.18%, matching the 92.27% predicted by theory. And a saga missing a compensating action left 200 orders charged but never fulfilled, with not a single error logged.

15.1 Microservices solve an organizational problem, not a technical one

The first thing to say clearly, because it decides everything that follows: splitting a system up does not make it faster. Section 15.2 proves the opposite with measurements. What microservices actually solve are people problems:

The last row is the most misunderstood. Very often the decision to "move to microservices" is really an attempt to fix a tangled codebase, and it never fixes it — because the boundaries you draw are only as good as your understanding of the domain at that moment, and if the code is already tangled that understanding usually is not good enough yet. The result is tangled code spread across a network: still tangled, now with debugging made harder.

This is also where Conway's law shows up: a system's structure tends to mirror the communication structure of the organization that built it. The practical consequence: if your organization has three teams with three clear responsibilities, three services is the natural shape; if your organization has one five-person team, eight services are just eight folders scattered across eight places — plus eight pipelines, eight dashboards, and eight on-call rotations.

15.2 The measured cost

The lab runs the same four-step use case two ways. mode=mono: the four steps are four function calls inside one process. mode=micro: the four steps are four HTTP requests to other instances. The amount of work per step is identical.

And remember the measurement conditions: three containers on the same machine, no switch, no crossing to a different availability zone. With Redis in Lesson 13 we already saw a Docker-internal network round trip cost about 0.07 ms while a real round trip inside a data center costs 0.5–2 ms. Apply that same order of magnitude here and four real hops would add several extra milliseconds, not fractions of one.

But latency is not the most expensive part. The most expensive part is cumulative availability: a chain only succeeds when every link succeeds, so with $n$ services each available with probability $p$, the chain is only available $p^n$ of the time.

$$P_{\text{chain}} = p^{\,n}$$

The lab injects 1% errors into every step and counts the chain's real end-to-end success rate:

The gap between measured and theoretical is under 0.1 percentage points at all four levels — the formula is not a rough estimate, it accurately describes what is actually happening. Translated into operational terms: five services each at 99.9% give you a chain of $0.999^5 \approx 99.5\%$, meaning downtime grows from about 43 minutes a month to about 3.6 hours a month — with no single service ever violating its own SLO.

15.3 Crossing a service boundary means losing ACID

In a monolith, "create the order — charge the card — reserve the stock" lives inside one transaction: either all three happen, or none of them do, and the database takes care of that. Once those three things belong to three services with three databases, that transaction no longer exists. Nothing at the infrastructure layer replaces it — you have to write the replacement yourself, and it is called a saga: a chain of local transactions, each step paired with a compensating action to undo it if a later step fails.

The lab builds a three-step saga across three real services, each holding its own data, then makes the third step fail 200 times in a row:

$ ./tools/micro-test.sh saga 200 1     # CO hanh dong bu
  chay 200 saga, hong o buoc 3 (inventory), compensate=1
  app1: {"order":0,"payment":0,"inventory":0} compensations=200
  app2: {"order":0,"payment":0,"inventory":0} compensations=200
  app3: {"order":0,"payment":0,"inventory":0} compensations=0

$ ./tools/micro-test.sh saga 200 0     # KHONG co hanh dong bu
  chay 200 saga, hong o buoc 3 (inventory), compensate=0
  app1: {"order":200,"payment":0,"inventory":0} compensations=0
  app2: {"order":0,"payment":200,"inventory":0} compensations=0
  app3: {"order":0,"payment":0,"inventory":0} compensations=0

Read the second result in business terms: 200 orders were created, 200 charges succeeded, and 0 times was inventory reserved. Two hundred customers paid for something that will never arrive. No exception was thrown, no error line was logged — every individual step reported success exactly because it did succeed. Only the whole is wrong, and no single service stands anywhere that can see the whole.

15.4 Where to cut

If you have decided to split, the next question decides success or failure: where the cut goes. There is exactly one principle worth remembering — cut along business bounded contexts, meaning by the work a part is responsible for, not by technical layer. And every service owns its own data: no service reads another service's tables directly.

15.5 The practical path, and when not to split

The lowest-risk path is almost always: start with a modularized monolith — one process, but with clear module boundaries enforced by tooling (dependency checks in CI, banning cross-imports). That boundary gives you most of the organizational benefit without paying a cent for network hops, sagas, or tracing. Then only split out of the process when you see a concrete signal:

• One part has scaling needs very different from the rest (image processing versus plain CRUD, say).
• Two teams are genuinely blocking each other on deploy — measured by the number of releases actually delayed, not by a gut feeling.
• One part needs a different runtime or different compliance requirements (card data, health data).
• One part has a different risk profile: you want it to be able to fail on its own, not take the rest down with it.

When you do split, use the strangler fig approach: put a routing layer in front (this is exactly the API gateway from Lesson 4), move one route at a time to the new service, and keep the rest inside the monolith. This lets you roll back a step at a time and there is never a "big bang" day. A full rewrite is the opposite: you have to guess every boundary correctly on the very first try, while the thing you are missing is exactly the understanding of those boundaries.

Reproduce the measurements yourself

cd blog/sysdesign/sysdesign-lab

# --- Cost of the network hop: same 4 steps, same amount of work per step ---
./tools/micro-test.sh cost
#   mono   p99 2.06ms · 18,887 req/s
#   micro  p99 8.25ms ·  3,744 req/s     (4.0x slower at the tail · throughput /5.0)

# --- Cumulative availability: each hop fails 1% of the time ---
./tools/micro-test.sh availability
#   1 hop  98.95%  (theory 99.00%)   p99  3.08ms
#   2 hop  98.08%  (theory 98.01%)   p99  4.50ms
#   4 hop  96.01%  (theory 96.06%)   p99  9.04ms
#   8 hop  92.18%  (theory 92.27%)   p99 16.52ms

# --- Saga: 200 failures at step 3 (reserve inventory) ---
./tools/micro-test.sh saga 200 1   # with compensation    -> order=0,   payment=0,   inventory=0
./tools/micro-test.sh saga 200 0   # without compensation -> order=200, payment=200, inventory=0
#   Read the last line in business terms: 200 orders charged, 0 orders had inventory reserved.

In summary

Microservices solve an organizational problem — independent deploys, clear ownership boundaries, scaling a hot part on its own — not a performance problem. The measurements say the opposite: same four-step use case, splitting it makes p99 4 times worse and throughput 5 times lower, on three containers on the same machine where the network hop is cheaper than reality.

The part that costs more than latency is cumulative availability, and it has an exact formula rather than a rough estimate: the measured success rate (98.95% / 98.08% / 96.01% / 92.18% for 1/2/4/8 hops) matches $0.99^n$ to within 0.1 percentage points. Five services each at 99.9% give you 99.5% — from about 43 minutes to about 3.6 hours of downtime a month, with no single service ever violating its own SLO.

Crossing a service boundary means losing ACID, and its replacement — the saga — only works if you actually write the compensating action. Skip it and the system reports no error at all: the lab left 200 orders charged and 0 times inventory reserved, with every single step reporting success. What catches that is a reconciliation job, not an error log.

On where to cut: cut along business bounded contexts, each service owns its own data, and accept deliberate data duplication. Cutting by technical layer or sharing one database produces a distributed monolith — paying the full cost of distribution while getting none of the benefit. And because the cost of waiting is linear while the cost of a wrong cut is a step function, a modularized monolith is the better bet whenever you are not sure.

Lesson 16 solves the problem this lesson just created: once a request crosses four services, how do you know which hop ate the time and where it broke. Without measuring that, you cannot catch a cascading failure — which is why observability has to come before Lesson 17.