Designing an API Worth Using
An API is a promise you can't take back — resources, pagination, idempotency keys, and versioning for clients you'll never meet.
🎯 By the end you'll understand
Why REST is a set of pragmatic constraints and not a religion; how to model resources instead of actions; why status codes are a contract your clients code against; when cursor pagination beats offset pagination (and why offset collapses at scale); how idempotency keys defend against the network's unreliability; and what versioning strategy lets you evolve a live API without burning your robot fleet. You'll design TraceLab's actual ingest endpoints and ship a "harmless" change that breaks a simulated client — on purpose, so you learn what "breaking" really means before your robots teach you.
KEY INSIGHTAn API is a contract, not an interface
A user interface lets humans adapt. An API has no such luxury — the client is a program, and programs don't forgive surprises.
Your robot fleet logs telemetry to TraceLab at 50 Hz. There are twelve robots. It is 2 a.m. and one robot is mid-run when you rename POST /runs to POST /experiments because it "reads better." The robot gets a 404, silently drops 800 datapoints, and the ablation study that took three days to set up is now unrecoverable. You did not change the behaviour — you changed the spelling. The robot didn't know about the new spelling. This is the lesson.
An API surface — every URL shape, every field name, every status code, every error response structure — is a binding promise to every client that exists or will exist. Clients are automated. They parse your responses mechanically, branch on your status codes, and retry on your error signals. When you change the shape, the old clients break silently or loudly, depending only on how defensively they were written. Good API design is therefore a discipline of forward-thinking constraint: pick shapes that can absorb change, and treat everything you publish as permanent.
CORE CONCEPTResources, not actions — the one REST idea that matters
REST (Representational State Transfer, Roy Fielding's 2000 dissertation) is frequently treated as theology. It is actually a set of architectural constraints. Only one of them materially affects day-to-day API design: think in nouns, not verbs. Your URLs should name things; the HTTP method supplies the verb.
| RPC-style (verbs in URLs) | Resource-style (HTTP verbs do the work) | Why it matters |
|---|---|---|
POST /createRun | POST /runs | HTTP method is the verb; URL is the noun |
GET /getMetrics?run=42 | GET /runs/42/metrics | hierarchy expresses ownership clearly |
POST /deleteRun | DELETE /runs/42 | idempotent semantics fall out naturally |
POST /updateRunStatus | PATCH /runs/42 | partial update without replacing the whole doc |
The resource model is not aesthetic preference. It creates predictability: a client who has learned your /runs collection can guess how /robots will behave. Caches and proxies understand GET is safe and cacheable, DELETE is idempotent, and POST is neither — because HTTP says so in its own spec, and your API inherits all that meaning for free. RPC-over-POST throws it all away.
That said: don't be a REST purist. Actions that don't map cleanly onto resources are fine as RPC endpoints — POST /runs/42/archive, POST /tokens/revoke. The rule is prefer nouns where they fit, not never use a verb. The moment you find yourself writing GET /checkIfRunExists, something has gone wrong.
CORE CONCEPTStatus codes as contract — what your client codes against
HTTP status codes are not decoration. Your client programs branch on them. A robot client that sees 500 might retry. One that sees 422 knows the payload is malformed and does not retry. One that sees 200 trusts the operation succeeded. When you lie about status codes — returning 200 with a body that says {"error": "not found"} — you corrupt every layer that reads the protocol: retry logic retries on success, load balancer health checks pass, monitoring sees no errors. The most common status code sins:
| Code | Correct meaning | Common misuse | Client consequence |
|---|---|---|---|
200 OK | Request succeeded; body contains the result | Returning errors in the body ({"error":"..."}) | Retry logic never retries; monitoring never alerts |
201 Created | Resource was created; Location header points to it | Using 200 for creates | Client can't tell a create from an update |
400 Bad Request | Client sent a malformed request; do not retry | Using 500 for validation errors | Client retries a bad payload forever |
401 Unauthorized | No valid credentials; go authenticate | Confusing with 403 | Client may loop trying to refresh a token it can't use |
403 Forbidden | Authenticated but not authorized | Returning 404 to hide resource existence | Client can't distinguish "not found" from "denied" |
404 Not Found | Resource doesn't exist at this URL | Using it for "no results in query" | Client thinks endpoint is gone, not that query returned empty |
409 Conflict | State conflict (duplicate, optimistic lock) | Using 400 for all errors | Client can't distinguish "bad data" from "race condition" |
422 Unprocessable | Syntactically valid but semantically invalid | Not using it at all | Client can't tell well-formed-but-wrong from malformed |
429 Too Many Requests | Rate limited; Retry-After header says when | Returning 503 for rate limits | Client retries immediately instead of backing off |
503 Service Unavailable | Server temporarily down; retry is appropriate | Using it for permanent errors | Client retries a request that will never succeed |
GET /runs?robot=WALL-E with no matching runs should return 200 with an empty list — {"items": [], "next_cursor": null}. The resource (the collection) exists; it just has no members. Returning 404 here forces clients to distinguish "endpoint missing" from "query empty" — a distinction they should never have to make.
CORE CONCEPTCursor vs offset pagination — and why offset dies at scale
Any endpoint that can return many items needs pagination. There are two dominant strategies, and the difference matters the moment your dataset grows.
Offset pagination (?page=3&limit=20 or ?offset=40&limit=20) translates directly to SQL: LIMIT 20 OFFSET 40. It is easy to implement and lets clients jump to any page. The problems are real and compound at scale:
- Database cost:
OFFSET Nrequires scanning and discarding N rows to reach the ones you want. Page 10,000 requires discarding 199,980 rows first. Most databases cannot avoid this. - Phantom reads: if a new run is inserted between page 1 and page 2 fetches, every subsequent page shifts by one. Items appear twice or disappear silently — a particularly fun bug for a robot telemetry system logging live runs.
- No stable total: the count changes as you paginate; "5 of 200 pages" becomes wrong the moment anything is inserted or deleted.
Cursor pagination (?cursor=eyJpZCI6NDJ9&limit=20) instead encodes a position — typically the last seen ID or timestamp — and the query becomes WHERE id > :cursor ORDER BY id LIMIT 20. This is a range scan on an indexed column: fast at any offset. The "next cursor" comes back in the response and is opaque to the client. The trade-off: you cannot jump to page 7. You can only move forward (and sometimes backward, with bidirectional cursors). For infinite-scroll UIs and streaming ingesters that consume all results sequentially, this is always the right choice.
TraceLab's metric ingest endpoint will use cursor pagination from day one — robot fleets stream thousands of datapoints per run, and "page 500" must be as fast as "page 1." The cursor is opaque base64-encoded JSON on the wire; clients treat it as a token, not a number.
CORE CONCEPTIdempotency keys — because networks fail mid-request
Here is the failure mode that wakes engineers at 2 a.m. A robot sends POST /runs/42/metrics with 50 datapoints. The server receives the request, writes the data to the database, and starts sending back 201 Created. The network drops the response. The robot never sees a reply. Its retry logic fires. It sends the exact same payload again. The server, which has no memory of the first request, writes the data again. Your ablation study now has duplicate datapoints at every retry boundary. If your retries happen three times, you have quadrupled the duplication.
The solution is an idempotency key: a client-generated UUID sent in a request header (Idempotency-Key: 550e8400-e29b-41d4-a716-446655440000). The server stores the key alongside the result. On retry, it finds the key, sees it has already processed this request, and returns the original response — without touching the database again. The operation is now idempotent even though POST is not.
import uuid, requests def ingest_metrics(run_id, metrics, *, base_url, token): # Generate key once; store it alongside the request so retries reuse it idem_key = str(uuid.uuid4()) payload = {"metrics": metrics} headers = { "Authorization": f"Bearer {token}", "Idempotency-Key": idem_key, # server uses this for dedup } for attempt in range(3): try: r = requests.post( f"{base_url}/v1/runs/{run_id}/metrics", json=payload, headers=headers, timeout=5 ) if r.status_code in (200, 201): return r.json() if r.status_code == 409: # already processed return r.json() # idempotent success if r.status_code < 500: raise ValueError(f"client error {r.status_code}") except requests.exceptions.Timeout: pass # retry with same key raise RuntimeError("ingest failed after 3 attempts")
You will read this phrase in distributed systems marketing. It is not achievable across a real network without idempotency keys on both sides. What you can achieve is at-least-once delivery with idempotent operations — which is functionally equivalent when the server deduplicates. The Kafka documentation calls this "effectively once" and is honest about the implementation cost. Be skeptical whenever a vendor claims "exactly-once" without explaining their deduplication mechanism.
Idempotency keys need a storage strategy on the server. The simplest: a separate table idempotency_keys(key, created_at, response_status, response_body) — the server checks the key before processing and stores the result after. Keys expire after a reasonable window (24 hours covers all sane retry scenarios). The window also bounds the storage cost: you're storing short-lived UUIDs, not full history.
CORE CONCEPTVersioning — how you evolve without burning clients
The worst moment in API design is the day you discover you designed something wrong and now you must change it — but clients in the field depend on the old shape. Versioning is your escape hatch. The tradeoff is not "do we version?" but "which versioning strategy taxes us least?"
| Strategy | Example | Pros | Cons |
|---|---|---|---|
| URL prefix | /v1/runs, /v2/runs | Explicit, easy to route, easy to deprecate | Duplicated routes; breaking changes force path churn |
| Accept header | Accept: application/vnd.tracelab.v2+json | Semantically cleaner (same resource, different representation) | Invisible in URLs, tricky to test manually, caches often ignore it |
| Query parameter | /runs?version=2 | Easy to add | Accidentally cacheable with wrong key; often forgotten in clients |
| No explicit versioning | Additive-only changes forever | Zero overhead | Requires perfect forward-compatibility discipline; one misstep breaks everyone |
The industry has mostly converged on URL prefix versioning for external APIs precisely because it is obvious and survives being pasted into a Slack message. Stripe, GitHub, and Twilio all use it. The cost is that you maintain v1 and v2 simultaneously during a transition window — typically 6-12 months, announced with deprecation headers and a sunset date.
What counts as a "breaking change"? More than you think:
- Renaming a field (
run_id→experiment_id) - Changing a field's type (
"timestamp": "2024-01-01"→"timestamp": 1704067200) - Removing a field from the response
- Changing pagination from offset to cursor (yes, even if cursor is better)
- Adding a new required request field
- Changing an error code for an existing condition
- Changing the URL of an existing endpoint
Non-breaking (additive) changes are safe in-version: adding optional response fields, adding optional request fields with defaults, adding new endpoints, adding new error codes for new conditions. The rule: old clients sending old payloads must receive old-shaped responses without error.
RFC 8594 defines a Sunset header: Sunset: Sat, 01 Jan 2028 00:00:00 GMT. Pair it with a Deprecation: true header and a Link: </v2/runs>; rel="successor-version". Any client that logs headers will surface the warning automatically. This is the professional way to sunset a version — give clients months of advance notice, in-band, without a separate email thread they won't read.
SEE ITAPI design linter — build TraceLab's endpoints and ship a "harmless" change
Below you'll design TraceLab's run and metric endpoints by choosing each field of the API contract. A live critic flags contract smells with the client-side consequence of each. When your design is clean, ship a "harmless" change — rename a field — and watch the robot fleet react.
The rename scenario is the hardest lesson: run_id and experiment_id carry identical data. The contract change is zero bytes of semantic difference. But the robot fleet parses the field by name, and every robot that relied on run_id silently drops the value and associates its metrics with null. The data is present on the wire. The client just can't see it. This is a breaking change — and without versioning, it is invisible until someone notices the dashboard is blank.
TRACELABTraceLab v2: designing the actual ingest API
TraceLab leaves the single machine this stage. The CLI becomes a client; a real HTTP server replaces the file-open-close dance. Here is the v2 ingest API design — every decision justified by a principle from this lesson:
# Collection: runs POST /v1/runs # create run → 201 + Location: /v1/runs/{id} GET /v1/runs # list → 200 + cursor pagination GET /v1/runs/{id} # fetch run → 200 or 404 PATCH /v1/runs/{id} # update status/tags → 200 DELETE /v1/runs/{id} # soft-delete → 204 No Content # Sub-resource: metrics POST /v1/runs/{id}/metrics # ingest batch → 201; requires Idempotency-Key GET /v1/runs/{id}/metrics # fetch → 200 + cursor; ?step_gte=N for filtering # Request / response shapes (ingest) # Request body: { "metrics": [ {"step": 100, "name": "loss", "value": 0.312}, {"step": 100, "name": "reward", "value": 14.2} ] } # Success response (201): { "inserted": 2, "run_id": "r_abc123" # stable: never renamed without version bump } # List response (200): { "items": [ ... ], "next_cursor": "eyJzdGVwIjoxMDB9", # opaque; null when exhausted "has_more": true } # Error response (all 4xx/5xx): { "error": { "code": "METRIC_BATCH_TOO_LARGE", "message": "batch exceeds 1000 metrics; split and retry", "request_id": "req_7f3a..." # for support correlation } }
The error shape deserves attention: a machine-readable code string (robots branch on it), a human-readable message (engineers read it), and a request_id that ties a failure to a specific server log line. When your robot fleet pages you at 2 a.m. with "METRIC_BATCH_TOO_LARGE," you have the request ID, you find the log line, you see the payload size. Without request_id, you have "something failed" and a timestamp that covers 10,000 log lines.
TraceLab v0's CSV logger had no locking — concurrent writers interleaved rows. The Idempotency-Key header on POST /v1/runs/{id}/metrics is the v2 answer: the server is the single writer; concurrent robot clients send requests that the server serializes. The lost-update race still exists (Lesson 3.7 will expose it), but its blast radius is now bounded to the server side — where transactions can fix it — rather than spread across twelve robot filesystems.
Token auth (Authorization: Bearer ...) is part of the v2 contract too, but its mechanism — token generation, storage, validation — belongs to Lesson 3.5. The point here is that the auth header is part of the API contract: a 401 means "go get a token," and that signal must be stable and correct across all endpoints or every client's token-refresh logic breaks.
WEAKNESSWhere API design bites
This is the lesson's central failure mode. Renaming a response field, moving a nested field up one level, switching from a string to an integer ID — these feel like refactors. To a human reading the documentation, they are. To a robot client parsing JSON with data["run_id"], they are KeyError at runtime, or silent None propagation, or a type mismatch three frames downstream. The worst failures are silent: the robot continues logging, but the metric is silently associated with null and never appears on any chart. You will not know until someone asks why WALL-E's ablation results are missing. Treat every field name, every field type, and every status code as permanent once published.
This antipattern — {"status": "error", "message": "not found"} on a 200 response — is common in older APIs and in developers who "just want to keep it simple." The costs: load balancer health checks pass (the endpoint looks healthy); monitoring sees no HTTP errors; retry logic written against status codes won't retry; client code that assumes 200 means success passes corrupted data downstream. HTTP's status codes are a shared protocol that proxies, caches, monitoring systems, and client libraries all speak. Returning 200 for failures is opting out of the entire observability infrastructure the protocol provides for free.
GET /runs returning all runs, with no pagination, seems fine with 200 runs. At 2,000,000 runs — after 18 months of a busy robot fleet — it returns a JSON blob that takes 40 seconds, kills the database with a full table scan, and OOM-crashes the client that tried to parse it into memory. Every list endpoint must be paginated from day one. Adding pagination later is a breaking change (old clients expected all results; new response has a subset). The migration cost is always paid; the question is whether you pay it at design time or at incident time.
KEY INSIGHTHow this connects
- HTTP: The Lingua Franca (3.1) — the method semantics, header structure, and status code table that this lesson uses as vocabulary. If the contract concepts here feel abstract, go read 3.1's wire-level anatomy first.
- Inside the Server: The Request Lifecycle (3.3) — next lesson: once the API contract is clear, we open the server process itself: how a listen socket turns into a parsed request, how thread pools work, and where your handler runs relative to every other request.
The v2 ingest API designed here will be the live wire for the rest of Stage 3. Lesson 3.4 builds the Postgres schema behind it; Lessons 3.5-3.6 add transactions and handle the concurrent write race that the API's idempotency keys defend against; Stage 5 monitors it in production. Every design decision you make or skip here — pagination strategy, idempotency support, field naming stability — is something you will personally live with during Stage 5's incident response exercise.
GLOSSARYKey terms
- Resource
- A noun-shaped thing the API exposes — runs, metrics, robots. URLs name resources; HTTP methods act on them.
- Idempotency key
- A client-generated UUID sent in a request header so the server can detect and deduplicate retries. Makes non-idempotent operations (POST) safe to retry across network failures.
- Cursor pagination
- Pagination strategy where the server returns an opaque token encoding the position of the last item, and the next request uses that token as a range-scan key. Stable and O(1) regardless of depth.
- Offset pagination
- Pagination strategy using
?page=N&limit=M, mapping to SQLOFFSET. Simple to implement; fails at scale because cost grows with page number and live insertions cause phantom reads. - Breaking change
- Any change to a published API that causes old clients to receive responses they cannot correctly parse — field renames, type changes, URL changes, removed fields, changed status codes.
- Additive change
- A change that old clients can ignore: adding new optional response fields, new optional request fields with defaults, new endpoints, new error codes for new conditions.
- Sunset header
- An HTTP response header (RFC 8594) announcing the date on which an endpoint or API version will be decommissioned, giving clients advance in-band deprecation notice.
- RPC-style API
- An API that puts actions in URLs (
/createRun,/deleteRun) rather than using HTTP methods on resources. Loses HTTP semantics and predictability, though it is not always wrong for truly action-shaped operations.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1A colleague proposes POST /getRun?id=42 because "it's simpler." What two things are wrong?›
First, the verb getRun is redundant — the HTTP method GET already means "retrieve." The URL should name the resource: GET /runs/42. Second, using POST for a retrieval operation throws away HTTP semantics: caches won't cache it, proxies won't treat it as safe or idempotent, and load balancer health checks behave incorrectly. The fix is GET /runs/42, which is also shorter.
2Your robot's telemetry ingest uses offset pagination. At run #5,000,000, page 200 is slow and the fleet is dropping datapoints. Why, and what fixes it?›
Offset pagination requires the database to scan and discard the first 200 × page_size rows before returning the ones you want. At scale this is a full table scan whose cost grows with depth. Additionally, live inserts shift pages, causing items to be skipped or duplicated between requests. The fix is cursor pagination: WHERE step_id > :cursor ORDER BY step_id LIMIT N — an indexed range scan that is O(1) regardless of how deep into the dataset you are.
3The network drops a robot's response to POST /v1/runs/42/metrics. The robot retries with the same payload. What bad thing happens without idempotency keys, and how do they fix it?›
Without idempotency keys, the server has no memory of the first request. It processes the payload again and inserts a second copy of all 50 datapoints. After N retries you have N duplicates. With idempotency keys, the client sends the same UUID in the Idempotency-Key header on every retry. The server checks the key before processing: if it has seen the key, it returns the stored response without touching the database. The operation is now effectively idempotent even though POST is not.
4You rename the response field run_id to experiment_id because it "reads better." Is this a breaking change? What is the client-side consequence?›
Yes, this is a breaking change even though the data content is identical. Robot clients parse the response by field name. Code written as data["run_id"] now gets a KeyError (in strict parsers) or silently returns None. The robot may continue logging without error, but every metric is associated with null instead of the run ID, and the data is effectively lost. The fix is to bump the API version (/v2/...) and run the old field name in parallel for a transition period, or never rename the field at all.