ProjekinAja. Software Engineering Stage 5 · Shipping & Running Lesson 5.5
● Stage 5 · Lesson 5.5

Cloud Primitives and the Bill

Compute, storage, IAM, and egress fees — what you're actually buying, what you're still on the hook for, and when a $20 VPS beats AWS.

⏱ ~18 min read ☁ Cloud economics 💸 Cost as architecture

🎯 By the end you'll understand

The six cloud primitives that outlive any provider — VMs, managed databases, object storage, load balancers, regions/AZs, and IAM — plus where the shared-responsibility line sits. You'll be able to read a cloud bill, anticipate the egress trap, and make a principled build-vs-buy decision for TraceLab's production stack. Most importantly: you'll understand why cost is an architectural property you design for, not a surprise you discover in your inbox on Monday.

KEY INSIGHTThe invoice is your architecture's honest review

The cloud bill doesn't lie. It shows you every assumption you made about traffic, data gravity, and idle capacity — and it charges you for every wrong one.

AWS, GCP, and Azure all publish pricing pages that read like legal documents for a reason: the pricing structure is the contract, and the contract encodes what these platforms consider valuable. Engineers who don't read the contract don't understand the architecture they're buying. The $30,000 weekend bills that circulate on Hacker News are not bad luck — they are architectures designed by people who didn't think about cost until it arrived.

🔑 The architectural truth

Cost is not a runtime property you tune later. It is a consequence of topology, data placement, and access patterns — and those are decided when you draw the architecture diagram. Put your database in a different region from your app server and you'll pay for cross-region transfer on every query, forever. Put your inference service behind a NAT gateway and you'll pay per-byte for egress you don't even see. The engineers who ship cheaply make cost-conscious choices at design time; everyone else optimizes in shame at 2 a.m.

CORE CONCEPTSix primitives that outlive every provider

The marketing names change — EC2, Compute Engine, Azure VMs — but the primitives beneath them have been stable for fifteen years. Learn these, not the console UIs.

1

Virtual Machines (instances)

A slice of a physical host: vCPUs, RAM, and ephemeral local disk. You own the OS up. The hypervisor and hardware are the provider's problem. Billed per second (AWS, GCP) or per minute — even when the instance is idle. Instance sizes follow a naming taxonomy: t3.medium, c6g.xlarge. "General", "compute-optimized", "memory-optimized", "GPU" describe the CPU:RAM:accelerator ratio. TraceLab's web server is a VM.

2

Object Storage (blob / bucket)

A flat namespace of key → bytes. No filesystem hierarchy — just prefixes that look like paths. Infinitely scalable, eventually consistent (S3 has had strong read-after-write since 2020), billed on capacity + requests + egress. Durability is the killer feature: S3 promises 11 nines by replicating across AZs automatically. Object storage is where TraceLab will put experiment artifacts: model checkpoints, logs, plots.

3

Managed Databases (RDS, Cloud SQL)

The provider runs the database software, handles backups, patching, failover, and replication. You provide SQL. You pay: instance size + storage + IOPS (often separately on RDS) + backup storage + data transfer out. The IOPS pricing is where the surprises hide — provisioned IOPS on AWS RDS can cost more than the instance itself at write-heavy workloads. Know your access pattern before you provision.

4

Load Balancers (ALB, NLB)

Distribute traffic across instances; terminate TLS; provide health-check-based routing; enable blue-green and canary deployments (the mechanism from Lesson 5.4). Billed on: fixed hourly + LCUs (load-balancer capacity units) on AWS, which are a function of connections, bandwidth, requests, and rule evaluations. In practice, ALBs cost $15–25/month at moderate traffic — cheap enough to always use, expensive enough to notice if you forget to delete the staging ones.

5

Regions and Availability Zones

A region is a geographic cluster of data centers (us-east-1, eu-west-2). An AZ is one isolated failure domain within a region — independent power, cooling, and physical building. Data transfer within an AZ is free; between AZs costs ~$0.01/GB (small, but it adds up at scale); between regions costs $0.02–0.09/GB. "Multi-region" is not free redundancy — it is a topology decision with a continuous data-transfer bill. Keep your app and its database in the same AZ unless you have a specific failure-model argument.

6

IAM (Identity and Access Management)

The access-control plane for everything in the cloud. Every API call is authorized against a policy attached to an identity (user, role, service account). IAM itself is free — the cost of getting it wrong is an exfiltration incident, not a bill. The shared-responsibility model says the provider secures the infrastructure; you secure what's in it. IAM is your side of that line. Next section.

Region (us-east-1) · AZ-a ALB load balancer VM (t3.small) TraceLab app Managed PG RDS / db.t4g.micro Object Storage S3 — artifacts, exports IAM who can call what API app role → S3 put only CI role → ECR push only no human keys in the app internet →
TraceLab's production stack mapped to the six primitives. IAM governs every API call; the region boundary is where egress charges start. The load balancer is the only resource touching the public internet.
📖 Managed vs self-run

Every managed service (RDS, ElastiCache, MSK) is a tradeoff: you pay 2–4× the equivalent EC2 cost for the instance, and in return you get automated backups, failover, patching, and monitoring. At small scale this is usually worth it — the engineering time to operate Postgres properly costs far more than the premium. At large scale (millions of queries/sec), the premium becomes the largest line item and self-managed becomes attractive again. TraceLab should use managed services until it has a database engineer on staff.

HOW IT WORKSIAM and the shared-responsibility line

The shared-responsibility model is the most important concept in cloud security, and it is also the most abused excuse for breaches. Here is the honest version:

LayerProvider's responsibilityYour responsibility
Physical infrastructureData center security, hardware, power, cooling, hypervisorNothing
Managed services (RDS, S3)Patching the DB engine, durability guarantees, hardware failuresAccess policies, encryption config, backup retention, network placement
Compute (EC2, containers)Hardware and hypervisor below the VMEverything from the OS upward: patching, firewall rules, user accounts, running software
IAMThe IAM service itself (its availability and correctness)Every policy you write, every role you create, every permission you grant
Network / VPCThe physical network fabricSecurity groups, NACLs, route tables, private subnets, VPC endpoints

The 2019 Capital One breach (100 million records) happened not because AWS had a vulnerability — the SSRF attack exploited a misconfigured WAF that allowed the application to query the EC2 instance metadata service, which handed back IAM credentials with excessive permissions. Their side of the shared-responsibility line had an open door. This is the canonical example because it is clean: the cloud worked as designed; the IAM policy was wrong.

💡 IAM best practices that actually matter

Least privilege, always. Your TraceLab app role needs S3 PutObject on exactly one bucket, not s3:*. No long-lived access keys in application code — use instance roles (EC2) or workload identity (GKE, EKS). Enable MFA on the root account and lock it in a drawer. Use separate roles for CI and for the running application — they have different access needs and different breach surfaces.

tracelab-app-role.jsonleast-privilege IAM policy
{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Effect": "Allow",
      "Action": [
        "s3:PutObject",         // write experiment artifacts
        "s3:GetObject",         // read them back for the UI
        "s3:DeleteObject"       // prune old runs (explicit opt-in)
      ],
      "Resource": "arn:aws:s3:::tracelab-artifacts-prod/*"
      // NOT s3:* on * — a compromised app can't exfiltrate other buckets
    }
  ]
}

The CI role for TraceLab needs exactly: ecr:GetAuthorizationToken, ecr:BatchCheckLayerAvailability, ecr:PutImage, and the ECS or EC2 deploy actions. It does not need database access, S3 access, or IAM permissions. These are separate roles because a compromised CI secret should have a blast radius, not a skeleton key.

SEE ITTraceLab cloud bill estimator

Cloud bills are the integration of your architecture over time. This estimator builds TraceLab's production stack from the six primitives and computes a monthly bill. Adjust size, storage, and traffic; switch between growth presets to expose the cost cliffs. The challenge: find a config that survives 10× traffic growth without 10× cost.

⚙ Interactive · Cloud bill estimator — TraceLab production stack

Provision each component and watch the monthly bill respond. Growth presets show the non-linear traps. All prices approximate AWS us-east-1 (2025 list pricing).

Monthly total
$0
Annual projection
$0
Biggest line item
—
Egress as % of bill
—

Configure your stack above. The challenge: find a config that survives 10× traffic growth without 10× cost increase.

The open-source-release preset is the important one. 10× traffic means 10× egress — which grows the egress line by 10× even if you haven't added a single byte of new code. Compute can scale (add instances; use a load balancer). Egress cannot be tuned away: it is determined by how much data your users actually download. If TraceLab serves large artifact files directly from S3 without a CDN, every download crosses the AWS egress meter. That is the architectural property you must address before you go viral, not after.

TRACELABChoosing TraceLab's production stack

TraceLab v4 is containerized (Stage 5.1–5.2) and deploying via CI/CD (5.3–5.4). Now we decide what it runs on. The build-vs-buy decisions are real architecture choices with real cost and operational consequences.

TraceLab v4 — production stack decision recordarchitectural decision
# COMPUTE
# Choice: t3.small on EC2, single AZ, with ALB in front
# Rationale: TraceLab is CPU-light (mostly IO); t3 burstable is appropriate.
# ALB enables canary deployment from 5-4 without change.
# NOT t4g (ARM) yet — CI pipeline must build ARM images first.

# DATABASE
# Choice: RDS PostgreSQL db.t4g.micro, 20 GB gp3, single-AZ
# Rationale: managed patching + failover worth 2× cost vs. self-managed.
# Single-AZ: we accept an AZ-failure window; TraceLab is not customer-critical.
# NOT io2: our write pattern (append-only run logs) doesn't hit IOPS limits.
# Watch: if IOPS credit depletion appears in CloudWatch, revisit before gp3→io2.

# OBJECT STORAGE
# Choice: S3 standard, one bucket per environment (prod/staging)
# Artifacts PUT from the app container (IAM role, not a key).
# Lifecycle rule: move to S3-IA after 30 days, Glacier after 90 days.
# Saves ~60% on storage cost for old experiment artifacts nobody reads.
# Egress mitigation: pre-signed URLs for artifact download (S3 → user directly).
# This routes large downloads AROUND the app server — no egress from EC2.

# NETWORK
# App and DB in the same AZ to avoid cross-AZ transfer charges on every query.
# VPC endpoints for S3 (so EC2→S3 traffic doesn't cross the NAT gateway).
# These two choices eliminate ~80% of potential surprise data transfer charges.

The lifecycle rule and the pre-signed URL pattern are the two most impactful cost choices here, and neither requires more infrastructure. The lifecycle rule cuts cold storage costs by 60% automatically. Pre-signed URLs mean that when a researcher downloads a 2 GB model checkpoint, those 2 GB leave S3 and are billed as S3 egress ($0.09/GB) rather than EC2 egress ($0.09/GB) — the rate is the same, but the bandwidth pressure moves off the app server. At small scale this doesn't matter; at lab-wide scale it's the difference between needing a larger instance and not.

💡 VPC endpoints: the zero-engineering cost cut

By default, traffic from your EC2 instance to S3 goes through the public internet (via your NAT gateway if in a private subnet), and you pay NAT gateway data processing fees ($0.045/GB) on top of egress. A VPC gateway endpoint for S3 routes traffic over the AWS private network and has no data transfer cost. It takes three minutes to create and requires no application changes. This is the canonical example of an architectural choice that looks like "ops work" but is actually free money.

IN PRACTICEThe egress trap and how it works

Egress pricing is the most discussed cloud pricing feature because it is the one most likely to produce a sudden large bill. The mechanics are simple once you see them: cloud providers charge for data leaving their network to the public internet, but not for data arriving. The asymmetry incentivizes you to store data in the cloud (free inbound), and makes it expensive to leave (costly outbound). This is not a conspiracy — it reflects real bandwidth costs on internet exchanges — but the scale can be surprising.

Transfer pathAWS rate (approx)TraceLab scenario
Internet → AWS (inbound)$0/GBRobot telemetry arriving at TraceLab
Within same AZ$0/GBApp server → database queries
Between AZs (same region)$0.01/GB each wayApp in AZ-a querying DB in AZ-b: $0.02/GB
S3 → EC2 (same region)$0/GBApp reading artifact from S3
EC2 → internet (egress)$0.09/GB (first 10 TB)Direct artifact download from app server
S3 → internet (egress)$0.09/GB (after 1 GB free)Pre-signed URL download — same rate, but no EC2 bottleneck
NAT gateway data processing$0.045/GBPrivate subnet EC2 → internet (in addition to egress)
Cross-region transfer$0.02–0.09/GBRunning staging in eu-west-2 while prod is us-east-1
⚠ The $30k weekend — what actually happened

The canonical "accidental cloud bill" stories share a structure: an application with no egress limit exposed a large dataset publicly (sometimes accidentally), was scraped or load-tested, and transferred hundreds of terabytes before anyone noticed. AWS does not rate-limit your bandwidth — your credit card is the rate limit. CloudWatch billing alarms and AWS Budgets with hard alerts at 80% of your monthly budget are not optional extras; they are the smoke alarm. No serious production setup should lack them.

The egress trap has a second, subtler form: inter-AZ transfer in a distributed system. A service mesh where every RPC crosses an AZ boundary pays $0.02/GB on every call. At a few thousand RPCs per second carrying kilobyte payloads, that is hundreds of dollars per month in invisible overhead. This is why AWS introduced AZ-aware load balancing (zonal shift, Availability Zone affinity routing) and why service meshes like Linkerd have locality-aware routing. The principle: keep data movement within an AZ unless you have a specific resilience reason to cross it.

COMPARISONWhen a $20 VPS beats AWS

AWS is not always the right answer. The honest comparison includes the cases where it loses.

DimensionAWS (or equivalent hyperscaler)Hetzner/DigitalOcean/Linode $20 VPS
Monthly cost$50–200 for TraceLab-scale (compute + DB + storage + egress)$20 flat — more RAM and disk than equiv. AWS for the price
Egress$0.09/GB after first GB20 TB/month included on most plans
Managed servicesRDS, ElastiCache, SQS, Lambda — all availableYou run Postgres, Redis, etc. yourself
Scaling ceilingEffectively unlimited, with autoscalingFixed; manual upgrade path
Global regions33 regions, hundreds of AZsLimited (Hetzner: 3 regions)
Compliance / enterpriseSOC 2, ISO 27001, HIPAA BAAs, FedRAMPVaries; generally no BAA
Operational burdenManaged services absorb a lotYou patch, you back up, you monitor
Best forVariable workloads, enterprise requirements, needing AWS-specific servicesStable workloads, personal/research projects, egress-heavy apps, cost-constrained stages

TraceLab's research context fits the VPS case at early stages: the traffic is low and predictable, there's no compliance requirement, and the egress-inclusive pricing means artifact downloads are effectively free. A Hetzner CX22 (2 vCPU, 4 GB RAM, 40 GB SSD, 20 TB egress) costs €3.79/month as of 2025 — less than the ALB alone on AWS. The right answer depends on where you are in the project lifecycle.

🧪 The switching cost

The risk of the VPS path is vendor-gravity in disguise. If you build TraceLab's notification system on SQS and your auth on Cognito, migrating to a VPS later means rebuilding those pieces. If you build on open standards (Postgres, SMTP, OIDC), the underlying compute is swappable. The Lesson 5.6 IaC approach helps here: declared infrastructure can target either provider if the primitives are abstracted correctly.

WEAKNESSWhere cloud economics bites engineers

✗ Treating egress as a runtime variable, not an architectural property

The most expensive egress mistakes aren't in production — they're in the design. Putting your S3 bucket in us-west-2 and your application in us-east-1 because "they're both on AWS" pays cross-region transfer on every object read. Putting your media-serving service behind EC2 instead of CloudFront pays egress and CPU cost for bytes that a CDN would serve cheaper. These aren't things you can tune later with caching headers; they require topology changes. Draw the data-flow diagram before you provision, and annotate every arrow with its transfer pricing.

✗ IOPS amnesia on managed databases

RDS gp3 gives you 3,000 IOPS included. That sounds like a lot until you have a write-heavy workload (experiment ingestion pipelines can be surprisingly write-intensive) and you start hitting IOPS throttling — which manifests as database latency, not an obvious error. The "fix" is to provision io2, which bills per IOPS and can cost more than the instance itself at 10,000+ IOPS. The right fix is to check your actual IOPS usage before migrating and to consider whether the write pattern can be batched. CloudWatch has a WriteThroughput metric. Read it before you hit the cliff.

✗ Idle resources and forgotten staging environments

The second-largest category of accidental cloud spend after egress is idle compute. A staging environment running 24/7 at the same instance size as production, a load test cluster that was never torn down, a development RDS instance provisioned six months ago — these accumulate. AWS Cost Explorer's "unused resources" report exists for a reason. Scheduled stop/start for non-production instances (stop at 18:00, start at 09:00) can cut staging costs by 65% without any architectural change. Enforce it via a tag-based Lambda or AWS Instance Scheduler.

KEY INSIGHTHow this connects

  • The Network from First Principles (1.8) — regions, AZs, and egress pricing are ultimately governed by where packets travel and how far; the latency numbers from Lesson 1.8 explain why cross-region transfer costs more than cross-AZ, and why your app and database belong in the same AZ.
  • Containers: What Docker Actually Is (5.2) — the container image you built there is the artifact that runs on the VM and ECR instance provisioned here; layer discipline and multi-stage builds directly determine the instance size you need and the ECR storage bill you pay.
  • Infrastructure as Code (5.6) — next lesson declares the stack we just described in Terraform/Pulumi; the primitives you learned here are the things IaC manages. You can't write correct Terraform for resources you don't understand.

The stage theme is "code on a laptop is a rumor." After this lesson, the production environment has a price tag and a responsibility boundary. The primitives you've named here are the things that will appear as line items on the bill and as resources in the IaC files. The next step — declaring them in code rather than clicking them into existence in a console — is how you make the architecture reproducible and auditable.

GLOSSARYKey terms

Virtual Machine (instance)
A software-emulated computer running on a cloud provider's physical host; you own the OS layer and everything above it.
Object storage
A flat key→bytes store (S3, GCS) with no filesystem hierarchy; billed on capacity, request count, and egress. Durability through automatic multi-AZ replication.
Managed database
A database service where the provider operates the DB engine, handles patching, backups, and failover; you provide SQL and configuration.
Egress
Data transferred out of a cloud provider's network to the public internet; typically billed per GB and the source of most surprise cloud bills.
Availability Zone (AZ)
One isolated failure domain within a region — separate power, cooling, and physical building. Data transfer within an AZ is free; between AZs is metered.
IAM (Identity and Access Management)
The cloud access-control plane; every API call is authorized against a policy attached to an identity. Your side of the shared-responsibility line.
Shared-responsibility model
The contract between cloud provider and customer defining who secures what. The provider secures the infrastructure; you secure everything you put on it.
Pre-signed URL
A time-limited URL that grants a specific S3 object access to the bearer without requiring AWS credentials; routes large downloads directly from S3 to the user, bypassing the app server.

ACTIVE RECALLCheck your understanding

Answer in your own words first, then reveal.

1Why is egress an "architectural property" rather than a runtime tuning knob?›

Egress charges are a direct consequence of topology and data placement decisions made at design time — which region stores the data, whether downloads go through an app server or directly from S3, whether AZs are mixed across services. You cannot reduce egress with a config flag after the fact; you have to move the data or change the routing topology. By the time the bill arrives, the architectural decision has already been made and paid for.

2What is the shared-responsibility line for a managed database like RDS?›

AWS handles: hardware, hypervisor, the database engine software, patching it, automated backups, and multi-AZ failover. You handle: access policies (who can connect, which IAM roles can call the RDS API), encryption configuration, backup retention period, network placement (which VPC, which security groups), and the data you store in it. A misconfigured security group that exposes RDS to 0.0.0.0/0 is entirely your side of the line.

3When would a $20 VPS beat AWS for TraceLab — and what's the risk of that choice?›

A VPS wins when traffic is low and predictable, egress is high (VPS plans include 20 TB/month; AWS charges $0.09/GB), there are no compliance requirements, and managed services aren't needed. The risk: if you build on AWS-specific services (SQS, Cognito, Lambda), migration later is expensive. The mitigant is building on open standards (Postgres, SMTP, OIDC) so the underlying compute is swappable — which is exactly what IaC and the lesson-5-6 approach encourages.

4What does a VPC gateway endpoint for S3 do, and why does it reduce your bill?›

A VPC gateway endpoint routes traffic from EC2 instances to S3 over the AWS private network rather than through your NAT gateway or the public internet. Without it, every S3 request from a private-subnet EC2 pays NAT gateway data processing fees ($0.045/GB) on top of any egress charges. With the endpoint, that path has no per-GB fee. It requires no application code changes and takes three minutes to create — a free cost reduction that everyone running EC2 in a private subnet should use.

Next: declare this entire stack in code — Infrastructure as Code →