ProjekinAja. Software Engineering Stage 4 · Architecture Lesson 4.2
● Stage 4 · Lesson 4.2

Monolith, Services, Serverless: An Honest Accounting

Microservices solve an organizational problem you don't have — architecture styles priced in latency, ops burden, and sleep.

⏱ ~22 min read 🏗 Architecture decision 🔥 High-stakes tradeoffs

🎯 By the end you'll understand

Why the modular monolith is the correct default architecture for most systems. What the distributed-systems tax actually costs, line-item by line-item. How Conway's Law predicts your architecture whether you plan it or not. What the distributed monolith is and why it is the worst possible outcome. Where serverless fits and where it fails. And — most practically — how to locate the actual crossover point where splitting a system starts to pay, and how far TraceLab is from that point.

KEY INSIGHTArchitecture is a cost, not a promotion

Microservices architecture is the right answer to a specific organizational problem: teams so large that shared deployment creates bottlenecks. If you don't have that problem, you're paying the distributed-systems tax for nothing.

The most dangerous word in software architecture is "modern." Microservices, serverless, event-driven — these get presented as the evolved forms of the inferior monolith, a natural progression that serious engineers make. This framing is wrong, and it has burned a generation of startups. Architecture styles are tools that each carry a cost. The distributed-systems tax is real, substantial, and largely invisible until you're paying it. The honest question isn't "should I be modern?" but "what problem am I solving, and what am I willing to pay?"

🔑 Hold onto this

Every time you add a network boundary between two components, you pay: latency becomes a distribution instead of a local call, partial failure becomes possible where it was impossible before, debugging requires correlation IDs and distributed tracing instead of a stack trace, and operations requires running, monitoring, and deploying multiple services instead of one. These costs are not hypothetical. They are denominated in engineer-hours per week, incident mean-time-to-recover, and, eventually, in sleep. The modular monolith — a single deployable unit with well-enforced internal boundaries — is the right default because it defers all of these costs until there is actual evidence that a specific boundary would benefit from the split.

CORE CONCEPTThe modular monolith: why it's the correct default

A monolith is a system deployed as a single unit. A modular monolith is a monolith whose internals are deliberately organized into modules with enforced boundaries — clear interfaces, restricted cross-module imports, isolated data — so it could be split later if needed without a rewrite. This is not a compromise between "real architecture" and "the toy approach." It is the thing that systems like Shopify (ran as a Rails monolith until they had very good reasons to break off specific services), Stack Overflow (still largely monolithic at enormous scale), and Basecamp were built on for years at high velocity.

What a well-structured monolith actually gives you:

1

In-process function calls instead of network hops

A cross-module call inside a monolith is a Python function call: nanoseconds, no serialization, no partial failure, a traceback that shows the full call chain. Compare that to a gRPC call to another service: ~1ms round-trip minimum, serialization/deserialization on both sides, a timeout budget, a retry policy, and a half-written distributed trace you'll look at at 2 AM.

2

A single ACID transaction across the whole system

One of the most underrated benefits. When "create experiment run" needs to write to runs, metrics, and artifacts atomically, a monolith with a single database makes that a three-line transaction. In a distributed system, you either accept eventual consistency (with all the edge cases), implement saga patterns (weeks of work), or use distributed transactions (which are slow and complex enough that most teams never get them right).

3

One deployment artifact to build, test, ship, and roll back

Deploying a monolith is one pipeline. Rolling back is one operation. Tracing a bug requires one log stream and one debugger. With five services, every one of these is multiplied by five — plus the coordination between them.

4

Refactoring crosses module boundaries freely

When a module boundary turns out to be wrong — and it will — fixing it in a monolith is a rename and a test run. In microservices, it's a multi-service API change, a versioning problem, a contract test suite, and a coordinated deployment across teams. The distributed system freezes its internal architecture in amber.

MODULAR MONOLITH · one deployable unit Ingest module Analysis module Dashboard module fn call fn call shared database one build one deploy one rollback
The modular monolith. Module boundaries are enforced in code — restricted imports, explicit interfaces — not by network topology. The benefit: all the conceptual clarity of separate services, none of the operational tax.

The key constraint that makes a modular monolith scale better than people expect: vertical scaling is vastly underrated. A single well-written Python service on a modern VM can handle thousands of requests per second. When you hit the limit — and most systems never do — you add instances behind a load balancer. Horizontal scaling of a monolith is one Kubernetes deployment spec. It is not glamorous. It works.

💡 The "it won't scale" objection

The most common objection to monoliths is scaling. The response from practitioners who have actually operated both: "You can always split a well-structured monolith later. You cannot easily merge a poorly-designed microservices mesh." The cost of splitting a module that has grown to need it is bounded. The cost of untangling a distributed system built on wrong service boundaries is not.

HOW IT WORKSThe distributed-systems tax: each line item

When you add a network boundary between two components, you are not just "adding a network call." You are signing up for a bundle of costs that compound with each additional service. Let's price them honestly.

Tax line itemIn a monolithPer network boundary addedSurprise for new adopters
Latency ~10–100 µs (local function call + possible DB query) +1–5 ms base per hop, p99 often 50–200 ms under load Latency compounds across call chains. 5 services in sequence with 5 ms each is 25 ms added — and that's before retries.
Partial failure A function raises an exception. Clean error, full rollback. Service A calls Service B which is down. Did the request reach B? Was it processed? Safe to retry? You need idempotency keys, retry budgets, circuit breakers, dead-letter queues. Each one takes a sprint to get right.
Observability One log stream, one traceback, one debugger. Distributed traces with correlation IDs, log aggregation across services, per-service metrics dashboards. A bug that crosses three services requires an OpenTelemetry setup, a trace store (Jaeger, Tempo), and knowing how to read a flame graph.
Data consistency ACID transactions. BEGIN / COMMIT. Done. Two-phase commit (slow, complex, rarely correct at scale) or sagas (weeks of design) or "just accept eventual consistency" (and debug the edge cases). "Eventually consistent" sounds fine until a user creates a run and immediately queries it and gets nothing back.
Operational surface One build pipeline, one deploy, one health check, one alert rule. N build pipelines, N deploys, N health checks, N × N inter-service contract tests, N × (N-1) / 2 potential connection failure modes. At 10 services, the ops surface is not "10× harder" — inter-service dependencies make it combinatorially harder.
Schema evolution Migrate the DB once. All callers are in the same codebase. Each service has an API contract. Breaking changes require versioned endpoints, dual-reads, and coordinated deploys across teams. The "just add a field" that took 5 minutes now requires a backwards-compatible schema version, an API version bump, and updating every consumer.
Local development python app.py Running all services locally means Docker Compose with 8 containers, or a Kubernetes dev cluster, or mocks that diverge from production. A new hire who can't run the full system locally can't contribute. "Run it against staging" is not a dev loop.

None of these are insurmountable. Large organizations — Netflix, Amazon, Google — have solved all of them. But they solved them with hundreds of engineers dedicated to platform infrastructure. The cost does not go away; it gets paid by someone. When you adopt microservices without that investment, you are saying "I will pay these costs personally, with ad-hoc solutions, under production pressure." That is the common outcome, and it is not fun.

CORE CONCEPTConway's Law: your org chart is your architecture

In 1967, Melvin Conway observed that "organizations which design systems are constrained to produce designs which are copies of the communication structures of those organizations." In practice: if you have four teams, you'll build four major subsystems. If you have one team, you'll build one system that may or may not be modularly organized.

This runs in both directions, and the second direction is the actionable one: you can use Conway's Law deliberately. Amazon's "two-pizza team" rule and their service-oriented architecture are the canonical example — each team owns a service, the service boundary is the team boundary, and teams coordinate through APIs instead of meetings. This is microservices solving an organizational problem: deployment coupling across large teams. The service boundary replaces the organizational interface.

📖 The inverse Conway maneuver

Thoughtworks coined the "inverse Conway maneuver": design your team structure to produce the architecture you want, rather than letting your architecture fossilize your current team structure. If you want a particular service boundary, staff a team against it. If you don't have that team, the boundary will erode — people will bypass it for convenience because there's no organizational force maintaining it.

For a small team or a solo engineer: Conway's Law is mostly warning, not prescription. You're one person. Your "communication structure" is a to-do list. Building microservices does not give you the organizational benefits they were designed for — it gives you all the costs and none of the autonomy, because there are no separate teams to be autonomous. This is the heart of why resume-driven architecture fails: it imports the solution to a problem that doesn't exist yet.

Small team You +1 Conway Monolith natural outcome Large org (4 teams) T1 T2 T3 T4 Conway svc-A svc-B svc-C svc-D team boundary = service boundary (intentional)
Conway's Law is not a bug — it's a lever. Big organizations use microservices because the team boundary IS the right service boundary. Small teams use monoliths for the same reason.

SEE ITArchitecture cost simulator

Architecture decisions are made with feelings and then defended with feelings. This simulator prices them in actual costs. The top panel shows total-cost-of-ownership curves across three architectures as you vary team size, traffic, deploy frequency, and domain coupling. The bottom panel lets you slice TraceLab apart service-by-service and watch the ops burden, latency, and failure surface compound per cut.

⚙ Interactive · Architecture cost simulator

Adjust the four sliders to match your project. Curves update live. Then try cutting TraceLab apart in the service-split mode — watch the cost counters.

ARCHITECTURE VERDICT
Computing…

✂ TraceLab service-split mode

TraceLab has four natural module boundaries. Toggle each split on to see the ops burden and failure-surface compound. You're starting with the modular monolith baseline.

Ingest API — log_run(), CSV writer
Query API — find_run(), dashboard reads
Auth service — user sessions, API keys
Notification worker — alerts, email on threshold
Services in flight
1
Extra deploys/week
+0
Min added latency
+0 ms
Failure modes added
0
SPLIT ASSESSMENT
All modules in one process. Zero distributed-systems tax.

Notice how the crossover between monolith and microservices only appears at team sizes well above 10 and at very high deploy frequency. Below that line, the monolith curve stays lower on every cost axis. TraceLab — even a production version serving a small robotics lab — sits well to the left of every crossover. The split verdict only flips for the notification worker at high traffic, because fire-and-forget async work is the one pattern where a queue-backed worker genuinely helps.

COMPARISONServerless: a third cost model, not a third architecture

Serverless (AWS Lambda, Google Cloud Functions, Cloudflare Workers) is worth separating from microservices conceptually. It is not "microservices but smaller" — it is a different cost model for compute: you pay per invocation instead of per running server, and the platform handles scaling, patching, and (most of) availability. This is genuinely useful for specific workloads.

Workload patternServerless fitWhy
Event-driven hooks (webhook receiver, S3 trigger)Good fitBursty, short-lived, infrequent. You pay nothing when idle.
Scheduled batch jobs (nightly report, cleanup)Good fitRuns once, completes, vanishes. No reason to pay for always-on.
Stateless API with wildly spiky trafficConditionalWorks until you hit cold-start latency in the p99, or the compute-per-request cost exceeds a reserved instance.
Long-running background workersPoor fit15-minute Lambda timeout, cold start overhead, and per-invocation pricing inverts at constant load.
Anything requiring shared in-process statePoor fitFunctions are stateless — state needs an external store. Every read/write adds latency and cost.
High-throughput, latency-sensitive pathsPoor fitCold starts (50–500ms for Python) violate any reasonable p99 SLA. Provisioned concurrency costs as much as just running a server.

The practical rule: serverless is excellent as the glue between systems (the webhook that triggers a queue, the nightly job that sweeps the database) and as the deployment model for genuinely stateless functions with bursty traffic. It is a poor choice as the primary application runtime because you end up re-inventing the monolith in function form, paying per-invocation, and fighting cold starts everywhere. The vendor ecosystem — DynamoDB, SQS, API Gateway — becomes a distributed monolith with a billing statement attached.

WEAKNESSThe distributed monolith: worst of both worlds

There is an architecture pattern more expensive than either a well-run monolith or a well-designed microservices system: the distributed monolith. It has all the operational complexity of microservices and all the coupling problems of a poorly-structured monolith. It is where most failed microservices migrations end up.

⚠ Signs you've built a distributed monolith

Services that cannot be deployed independently — every release requires a coordinated multi-service deploy in a specific order. Services that share a database directly (two services reading and writing the same tables, bypassing each other's APIs). Services with synchronous chains four or five calls deep where any single timeout cascades. If you see any of these, you have the costs of distribution without the benefits of autonomy.

How distributed monoliths are born: a team decides to "migrate to microservices," draws service boundaries on a whiteboard based on the existing module structure, and starts splitting. The problem is that the existing module structure was designed for in-process coupling — shared state, tight data access, deeply nested calls. Drawing a network boundary through those dependencies does not remove the coupling; it just makes it invisible until 2 AM. The correct sequence — which almost no one follows — is to fix the coupling first (establish clean module boundaries in the monolith, enforce them, make each module own its data) and only then consider extracting into separate services. Lesson 4-1 covers the first step. This lesson is about not skipping straight to the second.

⚠ Resume-driven architecture

The pattern has a name in the industry: engineers who want microservices on their CV adopt them for a project where they solve no real problem, pay all the costs, and blame "microservices" (rather than premature adoption) when the project struggles. The solution being oversold does not invalidate the tool — it indicts the failure to price the problem before selecting the tool. The widget above exists precisely to force that pricing conversation.

⚠ The org that splits first and structures never

A variant: the team splits into microservices early, and then — because the ops burden of N services is real — never does the module-boundary work within each service. You now have N services, each of which is a mini-monolith with no internal structure. When a cross-cutting concern appears, it gets copy-pasted across N codebases. When a schema changes, N repos need updating. This is the worst outcome: you paid the split cost and got none of the encapsulation benefit.

TRACELABTraceLab's architecture moment

TraceLab v2 works. The God module is behind us (that was Lesson 4-1's work). We have three reasonably clean modules: ingest, query, and dashboard. They communicate through defined interfaces. The database is shared but accessed through proper query methods, not raw SQL scattered everywhere. This is a modular monolith — and it is the right architecture for TraceLab right now, full stop.

What does "right now" mean? Let's be specific. The simulator above says the crossover from monolith to services doesn't pay until:

TraceLab serves a robotics lab. It has one to three users. Deploys happen when someone has a new feature. The answer is: we will not split services in this stage. What we will do is make sure the modular structure is clean enough that splitting later is a choice, not a rewrite.

The one legitimate candidate for extraction — not now, but when the need is real — is the notification worker. Asynchronous, fire-and-forget, independent failure mode: this is the pattern where a queue-backed worker earns its keep. Here is the interface we would use when we do extract it, showing how the monolith-to-service boundary looks if the module is structured right:

tracelab/notifications.pyv3 — structured for future extraction
# tracelab/notifications.py
# Today: runs in-process. Future: publish to a queue instead of direct call.
# The interface is the same either way — that's the point of the module boundary.

from dataclasses import dataclass
from typing import Protocol

@dataclass
class ThresholdAlert:
    run_id: str
    metric: str
    value: float
    threshold: float

class NotificationSink(Protocol):
    def send(self, alert: ThresholdAlert) -> None: ...

# In-process implementation — used today
class LogNotificationSink:
    def send(self, alert: ThresholdAlert) -> None:
        print(f"[ALERT] {alert.run_id}: {alert.metric} = {alert.value:.4f} > {alert.threshold}")

# Queue-backed implementation — swap in when we need async
class QueueNotificationSink:
    def __init__(self, queue_url: str): self.queue_url = queue_url
    def send(self, alert: ThresholdAlert) -> None:
        # publish to SQS / Redis / whatever — the caller doesn't change
        ...

# Wired at startup via dependency injection — not hardcoded
def check_threshold(run_id: str, metric: str, value: float,
                      threshold: float, sink: NotificationSink) -> None:
    if value > threshold:
        sink.send(ThresholdAlert(run_id, metric, value, threshold))

Notice the structure: a Protocol (interface), an in-process implementation, a queue-backed implementation that satisfies the same interface, and dependency injection at the call site. When we extract the notification worker to a real service, the caller does not change. This is what "structured for future extraction" means — not premature splitting, but deliberate interface design. The cost of the interface discipline is small. The cost of retrofitting it onto a tightly-coupled monolith when you need to extract is large.

The stage milestone for TraceLab v3 tracks this philosophy across all four modules: clean boundaries, the v0 CSV debt paid off with a migration, a cache in front of reads, an idempotent ingest queue when load demands it — but all of this inside a single deployable unit until there is a specific, measurable reason to split.

WEAKNESSWhere this bites

✗ Splitting before the module boundary is clean

The most expensive mistake: extracting a service before enforcing the module boundary it represents. Two components that call into each other's internals, share a database table directly, or have circular dependencies do not become cleanly separated services by adding a network call between them. They become a distributed monolith. The prerequisite for service extraction is clean module boundaries — the work in Lesson 4-1 — not the other way around.

✗ "We'll need to scale anyway so let's start with services"

Premature optimization for a scale you have not reached, may never reach, and — critically — cannot anticipate the shape of. The component that will need independent scaling is almost never the one you guessed. Amazon's infamous "Bezos API Mandate" came after years of running a monolith. They knew exactly which services needed to scale independently because they had years of real load data. You don't. The modular monolith preserves optionality; the premature split freezes a guess.

✗ Ignoring operational cost until it hits you

Teams adopt microservices in a greenfield sprint with a clean CI/CD setup and no incidents. Six months later, they are running ten services, none of which has complete distributed tracing, two of which share a database that nobody admits is shared, and every incident requires five engineers to reconstruct a timeline from five log streams. The operational cost of distributed systems is not front-loaded — it accrues slowly and then compounds. The correct time to price it is before the adoption decision, not during the first major incident.

KEY INSIGHTHow this connects

  • Modules, Boundaries, and Coupling (4.1) — the module discipline that must precede any service extraction. Clean internal boundaries are the prerequisite. If you skipped that lesson for this one, go back.
  • Caching: The Two Hard Problems, Live (4.3) — the next architectural decision in TraceLab v3. A cache between the query module and the database is one of the cheapest wins a monolith can get — and avoids one of the most common premature splitting reasons ("the query path is too slow").

The thread through Stage 4 is deliberate constraint: every architectural element added only when the monolith demonstrably hurts. This lesson established the baseline — the modular monolith as the honest default — and the pricing framework for when to deviate from it. The rest of the stage adds specific mechanisms (caching, queues, failure math, security) each time the monolith actually shows the pain that mechanism relieves.

GLOSSARYKey terms introduced

Modular monolith
A single deployable unit with deliberately enforced internal module boundaries — clean interfaces, isolated data, restricted cross-module imports. The benefits of service separation without the distributed-systems tax.
Distributed-systems tax
The bundle of costs introduced by every network boundary: latency variance, partial failure modes, observability complexity, consistency challenges, and operational surface area.
Conway's Law
Organizations design systems that mirror their communication structure. Practically: service boundaries should match team boundaries, or one will drag the other toward it anyway.
Distributed monolith
A system split into separate services but still tightly coupled — services that cannot be deployed independently, share databases directly, or have deep synchronous call chains. All costs, no benefits.
Partial failure
A failure mode unique to distributed systems: Service A's call to Service B may fail in ambiguous ways — request sent but not processed, partially processed, or processed but the response lost. Requires idempotency, retries, and circuit breakers to handle correctly.
Saga pattern
A technique for maintaining data consistency across distributed services without two-phase commit: a sequence of local transactions, each publishing an event that triggers the next, with compensating transactions to undo completed steps on failure.
Cold start (serverless)
The latency penalty paid when a serverless function is invoked without a warm instance available. For Python runtimes, 50–500 ms. Catastrophic for latency-sensitive paths.
Inverse Conway maneuver
Deliberately designing team structure to produce the architecture you want, rather than letting existing org charts freeze accidental architecture in place.

ACTIVE RECALLCheck your understanding

Answer in your own words first, then reveal.

1What organizational problem do microservices solve, and why does that mean they're wrong for most small teams?›

Microservices solve deployment coupling across large teams: when teams are so large that a shared codebase and shared deployment pipeline creates bottlenecks, service boundaries let each team own, deploy, and scale their service independently. A small team does not have this problem — there is no deployment coupling because everyone can coordinate directly. Adopting microservices without this problem means paying all the distributed-systems tax (latency, partial failure, ops overhead, observability complexity) for none of the organizational autonomy that justifies it.

2What is the distributed monolith, and how does it emerge from a microservices migration?›

A distributed monolith is a system split into separate services that remain tightly coupled — services that share databases directly, cannot be deployed independently, or have deep synchronous call chains. It has all the costs of distribution and none of the benefits of autonomy. It emerges when a team splits services along existing module lines without first enforcing clean module boundaries — the coupling was already there, in-process; drawing a network boundary through it made it invisible and dangerous, not gone.

3What does Conway's Law predict will happen if a small team builds microservices?›

Conway's Law says systems mirror the communication structure of the organization that builds them. A small team's communication structure is tight and low-overhead — everyone talks to everyone. Building microservices doesn't change that; it means the team will maintain the service boundaries through the communication overhead of a small team (i.e., everyone calling into everyone else's services). The result is either a distributed monolith (tight coupling across network boundaries) or constant coordination overhead that provides no velocity benefit over a single shared codebase.

4Name three specific cost line items from the distributed-systems tax that don't exist in a monolith.›

Any three from: partial failure (ambiguous failure modes requiring idempotency and retry logic); distributed tracing (observability across service boundaries requires correlation IDs, a trace store, and tooling that doesn't exist in a monolith); data consistency challenges (ACID transactions across services require sagas or two-phase commit instead of a BEGIN/COMMIT); inter-service contract versioning (API changes require backwards compatibility and coordinated deploys); cold-start latency (if serverless); local development complexity (running all services locally requires Docker Compose or mocks).

Next: caching — the cache in front of the dashboard, and why "cache everything" is the second-most expensive mistake →