HTTP: The Lingua Franca
Methods, status codes, headers, and statelessness — reading raw HTTP fluently is a superpower for debugging everything web-adjacent.
🎯 By the end you'll understand
The raw anatomy of an HTTP request and response — method, headers, body, status — and why getting any one of them wrong is the source of most "API bugs." You'll understand statelessness and what it demands of every layer above it, how caching headers and cookies actually work at the byte level, what HTTP/2 and HTTP/3 improve (and why that doesn't change the semantics you need to know), and how to read a raw exchange and immediately spot the mistake. TraceLab v2 starts here: a real HTTP API for logging runs.
KEY INSIGHTMost API bugs are HTTP-semantics bugs
Your "API client" is a thin wrapper around HTTP. When it breaks, the bug is almost always in the HTTP layer — wrong method, wrong Content-Type, wrong status interpretation — and you can see it if you look at the raw bytes.
You've called APIs, seen 401s and 422s, parsed JSON, and written fetch() calls. Here is the uncomfortable question: can you write out a syntactically correct HTTP request from memory? Can you tell, without a library, what your client is actually sending? If the honest answer is no, you're debugging with a blindfold on — and in production, the blindfold is always where the bug is. Every REST wrapper, every HTTP client library, every framework is ultimately formatting and parsing the same text protocol that Tim Berners-Lee first described in 1989–1991 (the first formal specification, RFC 1945, arrived in 1996). You cannot debug an abstraction you've never seen naked.
HTTP is the assumed substrate of the modern web, which means every tool tries to hide it from you. Frameworks route paths. ORM adapters speak JSON. But the moment something goes wrong — 415 Unsupported Media Type, CORS preflight rejection, a cache hit returning stale data — the hiding stops being helpful. The engineers who resolve these in minutes are the ones who can read raw HTTP. This lesson gives you that fluency deliberately, including making the exchanges go wrong on purpose, so the error vocabulary is familiar before you meet it at 2 a.m.
HOW IT WORKSRequest anatomy: what actually goes over the wire
HTTP is a text protocol (for 1.1; 2 and 3 are binary but express identical semantics). A client opens a TCP connection and sends a message that looks exactly like this:
POST /api/runs HTTP/1.1 ← request line: method SP path SP version CRLF Host: tracelab.local ← mandatory since HTTP/1.1 (virtual hosting needs it) Content-Type: application/json ← tells the server what you sent Accept: application/json ← tells the server what you'll accept back Content-Length: 52 ← byte count of the body (exactly 52, or things break) ← blank line (CRLF) separates headers from body {"run_id":"exp042","loss":0.31,"steps":500}
Three parts: a request line, headers (one per line, Name: value), and an optional body separated from the headers by a blank line. The server's response mirrors the structure:
HTTP/1.1 201 Created ← status line: version SP code SP reason CRLF Content-Type: application/json Content-Length: 42 Location: /api/runs/exp042 ← RFC 7231: 201 SHOULD include Location {"id":"exp042","created_at":"2026-06-12"}
That's it. Nothing more. Every framework, every SDK, every "REST client" is a machine that serializes into this format and parses the response back out. The protocol's beauty — and its danger — is that it's so simple that you can get most things right by accident. The failure modes live in the details.
HTTP/1.1's statelessness means any server replica can handle any request — the horizontal scaling story of the entire web. The price: you carry state yourself, in cookies, tokens, or query params. This bargain is the source of "session" as an application-layer invention and the reason authentication is non-trivial. When TraceLab v2 adds token auth in this stage, you'll feel the bargain directly.
CORE CONCEPTMethods are contracts, not suggestions
The HTTP method isn't just a label — it carries a contract with defined semantics that every intermediate (proxy, CDN, browser cache) is allowed to rely on. Breaking the contract silently works in development and spectacularly fails in production.
| Method | Safe? | Idempotent? | Body? | Semantic | Common mistake |
|---|---|---|---|---|---|
GET | Yes | Yes | No* | Retrieve a resource. Must have no side effects. | Using GET for writes because it's "simpler." Caches will repeat it freely. |
POST | No | No | Yes | Create a new resource or trigger an action. Response may differ each call. | Using POST for idempotent creates — two clicks = two rows. |
PUT | No | Yes | Yes | Replace the target resource entirely with the payload. | Using PUT when you mean PATCH — nuking fields you didn't intend to touch. |
PATCH | No | No† | Yes | Partial update. Only the included fields change. | Sending the full object — correct behavior, but wastes bytes and hides intent. |
DELETE | No | Yes | Rarely | Remove the resource. Safe to retry: deleting a deleted resource is still 200/204. | Returning 404 on repeat DELETE — breaks idempotency, breaks retry logic. |
HEAD | Yes | Yes | No | Identical to GET but no response body. Use for existence checks and cache validation. | Barely used by developers; very useful for CDN cache busting checks. |
OPTIONS | Yes | Yes | No | Ask server what methods/headers are allowed. The CORS preflight uses this. | Forgetting to handle OPTIONS — CORS breaks for every cross-origin POST. |
Two properties matter more than the method name. Safe means the server's state does not change — browsers and caches use this to prefetch and replay freely. Idempotent means sending the same request N times has the same effect as sending it once — crucial for retry logic. A GET request that triggers a side effect (say, logging a "view" to your analytics) violates safety; a POST that creates rows is not idempotent, so retrying on network failure is dangerous without an idempotency key.
†PATCH is not required to be idempotent by the RFC. A JSON Patch document saying "increment by 1" is explicitly non-idempotent. Most real-world PATCH endpoints are idempotent in practice — but you cannot assume it.
CORE CONCEPTStatus codes are machine-readable intent
The three-digit status code is an API contract between the server and every intermediary — proxies, CDNs, client libraries — not just the human reading the response. The class of the code (the first digit) is often all an intermediary acts on; the specific code is what your application logic should switch on. Using 200 OK for everything is technically valid and operationally disastrous.
| Class | Meaning | Key codes | The failure mode |
|---|---|---|---|
| 1xx | Informational | 100 Continue, 101 Switching Protocols | Rarely matters — until you implement WebSockets and forget 101. |
| 2xx | Success | 200 OK, 201 Created, 204 No Content | Returning 200 with {"error": "..."} in the body — the client has no idea it failed. |
| 3xx | Redirection | 301 Moved Permanently, 302 Found, 304 Not Modified | Returning 301 when you mean 302 — browsers cache the permanent redirect forever. Hard to undo. |
| 4xx | Client error | 400 Bad Request, 401 Unauthorized, 403 Forbidden, 404 Not Found, 405 Method Not Allowed, 409 Conflict, 422 Unprocessable Entity | Returning 500 for client mistakes — retries that can never succeed, forever. |
| 5xx | Server error | 500 Internal Server Error, 502 Bad Gateway, 503 Service Unavailable, 504 Gateway Timeout | Returning 500 for all errors — client can't distinguish transient from permanent, so retry logic guesses. |
401 Unauthorized means "I don't know who you are; send credentials." 403 Forbidden means "I know who you are, and you're not allowed." Confusing these leaks information: returning 403 when the resource doesn't exist tells an attacker that it exists but they can't see it. Many APIs return 404 instead of 403 for that reason — a deliberate semantic lie for security purposes.
The 3xx family deserves a moment. 304 Not Modified is the caching system's heartbeat — the server confirms that the client's cached copy is still fresh, and the response has no body. Understanding 304 is understanding half of how CDNs work. 301 Permanent is a warning to be wary of: browsers (and curl with -L) follow it, and the redirect destination is cached indefinitely. Changing your mind later requires invalidating every client that visited — which in practice means you can't.
SEE ITBuild and break HTTP requests
The widget below simulates a TraceLab API server. Compose a request field by field — method, path, Content-Type, Accept, Authorization — then fire it and see the raw bytes both directions. The deliberately wrong combinations produce the error vocabulary before you meet it in production. Read the response carefully: the status code, the error body, and the hint. The server logic is deterministic and correct per the RFCs.
Three patterns worth deliberately triggering. First, POST to /api/runs with text/plain Content-Type: the server returns 415 Unsupported Media Type because it only speaks JSON. Second, GET to /api/runs/exp042 with a valid ETag: the server returns 304 Not Modified with no body — that's the caching system working correctly. Third, any request with the "missing" token: the server returns 401 Unauthorized with a WWW-Authenticate header telling you exactly what kind of credential to send. Each of these is an HTTP semantic, not an "API bug."
IN PRACTICEHeaders that change everything
Headers are the programmable metadata of HTTP — they modulate caching, authentication, content negotiation, and connection management. A misconfigured header is often invisible in a happy path and catastrophic in an edge case.
Authentication headers
Authorization: Bearer <token> is the modern idiom. The server must validate the token on every request — statelessness means it cannot remember your previous authentication. The WWW-Authenticate response header is the protocol's way of telling a client exactly which scheme to use: WWW-Authenticate: Bearer realm="tracelab". Most developers never send this; most clients ignore its absence. Both are wrong.
Caching headers — the ones you're actually missing
Cache-Control: max-age=3600 tells every cache in the chain to serve this response without contacting the origin for 3600 seconds. ETag: "abc123" is a fingerprint of the response body; the client sends it back as If-None-Match: "abc123" and gets a 304 if nothing changed. Cache-Control: no-cache means "always validate with the origin before serving" — not "never cache." This confusion is the source of many "my cache isn't working" tickets.
Content negotiation headers
Content-Type: application/json tells the server what format you sent. Accept: application/json tells the server what format you want back. A server that ignores Accept is technically violating the spec but extremely common. A server that returns a different Content-Type than it advertised is the source of JSON parsing errors that appear to be client-side bugs.
CORS headers — the browser-only pain
CORS is enforced by browsers, not servers or HTTP. When a browser makes a cross-origin request, it first sends an OPTIONS preflight asking what's allowed. The server must respond with Access-Control-Allow-Origin, Access-Control-Allow-Methods, and Access-Control-Allow-Headers. If any is missing, the browser blocks the real request. The server never saw anything wrong — the client was blocked by its runtime. This is why "it works in curl but not the browser" is almost always a CORS problem.
Cookies are set by the server with Set-Cookie: session=abc; HttpOnly; Secure; SameSite=Strict. Each flag matters: HttpOnly prevents JavaScript from reading the cookie (XSS mitigation). Secure requires HTTPS. SameSite=Strict prevents the cookie from being sent on cross-site requests (CSRF mitigation). Leave any of these off and you've introduced a vulnerability class. Leaving all of them off and shipping to production is not hypothetical — it happens constantly.
COMPARISONHTTP/1.1 vs 2 vs 3 — without the hype
All three versions express the same application semantics: methods, headers, status codes, bodies. The differences are entirely in the transport — how bytes move over the wire. Knowing this prevents two failure modes: cargo-culting HTTP/2 as a semantic change, and dismissing HTTP/3 as marketing.
| Version | Transport | Key improvement | Gotcha |
|---|---|---|---|
| HTTP/1.1 RFC 7230 (2014) | TCP, text framing | Persistent connections (keep-alive). Before this, each request opened a fresh TCP socket — ruinously slow. | Head-of-line blocking: a slow response blocks all subsequent requests on the same connection. Clients worked around this by opening 6 connections per domain — a browser hack, not a spec feature. |
| HTTP/2 RFC 7540 (2015) | TCP, binary frames, multiplexed streams | Multiple requests in flight simultaneously over one TCP connection. Header compression (HPACK). Server push (rarely used correctly). | TCP head-of-line blocking remains: a single dropped packet stalls all multiplexed streams because TCP must deliver in order. Server push was complex, widely misused, and has been removed from Chrome. |
| HTTP/3 RFC 9114 (2022) | QUIC (UDP), binary frames | Eliminates TCP head-of-line blocking at the transport layer. Each QUIC stream is independent — a lost packet in one stream doesn't stall others. Faster connection setup (0-RTT resumption). | QUIC runs over UDP; some corporate firewalls block it. Falls back to HTTP/2. TLS 1.3 is mandatory. The QUIC specification alone is 151 pages — the "simple UDP" narrative is a simplification. |
The practical takeaway: HTTP/2 is the right default for internal APIs today — every major server supports it, and the multiplexing wins are real on networks with any latency. HTTP/3 matters for public-facing endpoints on mobile networks where packet loss is common, and is handled automatically by most CDNs. Your application code doesn't change either way — the semantics are identical.
TRACELABTraceLab v2 begins: HTTP API skeleton
Stage 3 milestone: TraceLab leaves the single machine. The CLI stays, but it becomes an HTTP client calling a real server. This lesson you write the API skeleton — the interface contract — before the server internals exist. That's the right order: agree on HTTP semantics, then build the implementation.
The resource model is simple. A run is the record TraceLab logs. The API surface for Stage 3:
| Method | Path | What it does | Correct status |
|---|---|---|---|
| POST | /api/runs | Create a new run record | 201 Created + Location header |
| GET | /api/runs | List recent runs (paginated) | 200 OK |
| GET | /api/runs/{run_id} | Fetch one run by ID | 200 OK or 404 Not Found |
| PATCH | /api/runs/{run_id} | Append metrics mid-run | 200 OK |
| DELETE | /api/runs/{run_id} | Remove a run (idempotent) | 204 No Content (even if already gone) |
Here's the FastAPI skeleton with correct HTTP semantics — no database yet, just in-memory state, because this lesson is about the HTTP contract, not persistence:
from fastapi import FastAPI, HTTPException, Response, status from pydantic import BaseModel from typing import Optional, Dict import time, uuid app = FastAPI() db: Dict[str, dict] = {} # in-memory; Stage 3.4 swaps in Postgres class RunCreate(BaseModel): run_id: str metrics: dict = {} @app.post("/api/runs", status_code=201) def create_run(body: RunCreate, response: Response): if body.run_id in db: raise HTTPException(409, "run_id already exists") # Conflict db[body.run_id] = {"run_id": body.run_id, "metrics": body.metrics, "created_at": time.time()} response.headers["Location"] = f"/api/runs/{body.run_id}" # RFC 7231 §7.1.2 return db[body.run_id] @app.get("/api/runs/{run_id}") def get_run(run_id: str): if run_id not in db: raise HTTPException(404, f"run '{run_id}' not found") return db[run_id] @app.delete("/api/runs/{run_id}", status_code=204) def delete_run(run_id: str): db.pop(run_id, None) # idempotent: missing key is fine # 204 = no body; don't return anything
Two design decisions worth explaining. The DELETE handler uses db.pop(run_id, None) — discarding the KeyError — because DELETE should be idempotent per RFC 7231. Retrying a DELETE on an already-deleted resource must also succeed. Returning 404 on the second call breaks retry logic in every client. The POST handler returns 409 for duplicate run_id rather than silently succeeding or overwriting — that's the correct conflict signal, and it's what lets the client decide whether to retry with a different ID or merge. These are HTTP semantic decisions, not implementation details.
The old tracelab.py log command will call POST /api/runs instead of appending to a CSV. The CSV was local state; the API is shared state. That shift — from file append to HTTP POST — is the architectural inflection point of Stage 3. The telemetry data starts flowing across a network, which is what makes concurrent writers and race conditions suddenly real.
WEAKNESSWhere HTTP semantics bite hardest
The most common sin: returning {"status": "error", "message": "not found"} with a 200 status code because "it's easier." Every HTTP client, CDN, and retry library classifies 2xx as success. Your error disappears into a cache hit. Your monitoring alert never fires. The request gets retried indefinitely. Status codes exist so that every intermediary — not just your code — can route correctly. A 404 means "don't cache this." A 503 means "retry with backoff." A 200 means "cache me and never check again." Returning 200 for errors hands that decision to no one.
GET is safe — caches may store and replay it freely. Put a side-effectful operation behind GET because it's "simpler to call," and the CDN will happily replay your "send email" endpoint to every user who visits the cached URL. The reverse also bites: using POST for idempotent reads means you cannot use HTTP caching at all, even when the response is identical every time. Method choice is not aesthetic — it is a cache policy declaration.
Sending JSON but advertising Content-Type: text/plain is the source of countless 415 errors that developers call "API bugs." The inverse is worse: a server that returns HTML (its error page) but sends Content-Type: application/json, causing the client to try JSON-parsing an HTML error message and generating a confusing secondary parse error that hides the original 500. Read the Content-Type. Check that it matches. Your HTTP client library's body parser relies on it entirely.
KEY INSIGHTHow this connects
- The Network from First Principles (1.8) — TCP, IP, and sockets are the foundation HTTP is built on. A request line is just bytes going over the reliable byte stream TCP provides. If you haven't read 1.8, the "over TCP" in this lesson is a handwave — go back and fill it in.
- Designing an API Worth Using (3.2) — HTTP gives you the vocabulary; 3.2 is about how to use it to design an API that developers don't hate. REST constraints, resource modeling, versioning, and the real tradeoffs of OpenAPI specs come next.
HTTP is the protocol everything else pretends not to be built on. REST APIs, webhooks, GraphQL over POST, gRPC-web — they all ride HTTP. The stage theme is "systems you can dissect," and HTTP is the first system in Stage 3 you learn to read naked. Every lesson from here uses it as infrastructure; this lesson makes the infrastructure visible.
GLOSSARYKey terms introduced
- Request line
- The first line of an HTTP request:
METHOD path HTTP/version CRLF. The method, resource path, and protocol version all in one short declaration. - Status code
- The three-digit machine-readable integer in the response status line. The first digit (1–5) signals the class; intermediaries act on the class, your code acts on the specific number.
- Idempotent
- An operation where applying it N times has the same effect as applying it once. GET, PUT, DELETE, and HEAD must be idempotent per the HTTP spec; POST and PATCH need not be.
- Safe
- An HTTP method is "safe" if it is guaranteed to have no side effects on the server. Safe methods (GET, HEAD, OPTIONS) may be freely prefetched, cached, and replayed by browsers and proxies.
- Content negotiation
- The mechanism by which a client advertises which formats it can send (Content-Type) and receive (Accept), and the server selects accordingly. Mismatches produce 415 or 406 status codes.
- ETag
- An opaque identifier for a specific version of a resource, sent in the response. The client echoes it back as If-None-Match on subsequent requests; a 304 Not Modified means the resource hasn't changed.
- Head-of-line blocking
- In HTTP/1.1: later requests cannot be sent until the current one completes. In HTTP/2: a single dropped TCP packet stalls all multiplexed streams. HTTP/3 (QUIC) eliminates this at the transport layer.
- CORS preflight
- An OPTIONS request a browser sends before a cross-origin non-simple request, asking the server whether the actual request is allowed. The server's Access-Control-Allow-* headers determine the browser's decision.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1What's the difference between "safe" and "idempotent" in HTTP, and why do both matter for caching and retry logic?›
Safe means no side effects — browsers and caches can replay a safe request freely without asking you. Idempotent means N calls equals 1 call in effect — retry logic can safely resend after a network failure. GET is both: cacheable and retryable. POST is neither: don't cache it, and don't auto-retry without an idempotency key. DELETE is idempotent but not safe: retrying a delete is fine, but a CDN shouldn't prefetch it. Getting these wrong means caches replay things they shouldn't, or retry clients create duplicate resources on transient failures.
2A client sends a GET request with If-None-Match: "abc123" and receives a 304. What does that mean, and what should the client do?›
The server is confirming that the resource the client has cached — identified by the ETag "abc123" — is still current. The 304 response has no body. The client should use its cached copy as-is. This is the conditional GET caching protocol: the client avoids a full transfer, and the server avoids generating and sending a full response. The cost is one round trip; the benefit is zero bytes of body transfer when the resource is unchanged.
3Why does "it works in curl but not in the browser" almost always mean CORS, and what specifically is the browser checking that curl doesn't?›
curl sends HTTP requests without enforcing the same-origin policy — it's a command-line tool, not a browser runtime. Browsers enforce CORS: before a cross-origin non-simple request (e.g., POST with JSON), the browser sends an OPTIONS preflight asking the server if it allows the cross-origin request. The server must respond with Access-Control-Allow-Origin, Access-Control-Allow-Methods, and Access-Control-Allow-Headers matching the request. If any is missing, the browser blocks the request entirely — the server never receives the original POST. curl never sends the preflight, so it always succeeds even when CORS headers are absent.
4A TraceLab DELETE endpoint returns 404 when a run doesn't exist. What contract does this break, and what should it return instead?›
DELETE must be idempotent per RFC 7231. Returning 404 on a second DELETE breaks idempotency: the first call returns 204, the second returns 404 — two different outcomes. Any retry logic that checks for non-2xx responses will now fail the retry, even though the desired state (run deleted) was already achieved. The correct behavior is to return 204 No Content regardless of whether the resource existed — use db.pop(run_id, None) and don't look at the return value. The end state is the same whether the resource was there or not.