Concurrency I: Threads, Processes, and the GIL
Shared mutable state is the original sin — races, locks, deadlocks, and what the GIL actually does to your options.
🎯 By the end you'll understand
The difference between concurrency and parallelism and why conflating them causes bad decisions. Why threads sharing memory is simultaneously powerful and dangerous. What the CPython GIL actually serializes and what it doesn't protect you from. How locks prevent races but invite deadlocks. When multiprocessing is the right answer for CPU-bound Python. And the rule that saves more careers than any framework: don't add concurrency until something measurable demands it.
KEY INSIGHTShared mutable state is the original sin
Concurrency bugs are uniquely cruel: they hide under normal load, appear under peak load, vanish when you add logging to investigate, and leave no stack trace. The only reliable defense is not needing one.
The last lesson put a database between you and the data, and the database's transaction engine made interleaving anomalies manageable. This lesson removes that safety net. When two threads increment a shared counter, there is no rollback, no isolation level, no advisory lock you can ask a server for. There is just memory, two instruction pointers, and whatever you built. If you built it wrong, you'll be debugging a bug that doesn't reproduce reliably, doesn't log, and can corrupt production data silently for weeks.
A race condition is what happens when the correctness of a computation depends on the order or timing of uncontrollable events — specifically, which thread executes which instruction first. Race conditions exist at the level of machine instructions, not source lines. A line like counter += 1 compiles to three operations: load, add, store. Any interleaving that separates those three operations across two threads can corrupt the result. Databases taught you the same hazard in a forgiving environment. Here, the environment is not forgiving.
CORE CONCEPTConcurrency vs parallelism: the distinction that matters
These words are often used interchangeably in job postings and blog posts. They mean different things, and confusing them leads directly to writing code that uses the wrong primitive for the problem.
Concurrency is a property of a program structure: the program is composed of independently progressing tasks that could, in principle, overlap. A web server handling multiple requests is concurrent even if it runs on one core — the tasks are structured to interleave.
Parallelism is a property of execution: tasks are literally executing at the same time on different hardware. Parallelism requires multiple CPUs or cores; concurrency does not.
The practical implication: you can have race conditions on a single-core machine (the OS schedules between threads). And you can have a perfectly safe parallel program if the threads never share mutable state. The danger is not in the number of cores — it's in shared mutable state.
Go's designer Rob Pike put it well: "Concurrency is about dealing with lots of things at once. Parallelism is about doing lots of things at once." Concurrency is the design pattern; parallelism is one possible execution strategy. (Note: the phrase comes from a 2012 talk, not a paper — verify before citing.)
HOW IT WORKSThreads and shared memory: power and terror
A thread is the smallest unit of execution scheduled by the OS. Every process starts with one thread. You can spawn more. The key property is: threads within a process share the same address space — the same heap, the same global variables, the same open file descriptors. That sharing is the entire point (cheap communication, no IPC overhead) and the entire problem (no isolation, race conditions everywhere).
Compare this to processes, which each get their own private address space. A bug in one process cannot corrupt another's memory — the kernel enforces the boundary. Communication between processes requires explicit mechanisms: pipes, sockets, shared memory segments with manual synchronization, message queues. Heavyweight but safe.
| Property | Threads (same process) | Processes |
|---|---|---|
| Memory | shared — all threads see the same heap | isolated — each has private address space |
| Communication | direct reads/writes to shared variables | pipes, sockets, shared-mem + locks |
| Spawn cost | microseconds (stack + TLS only) | milliseconds (full address-space copy or fork) |
| Crash blast radius | any thread can corrupt all shared state | crash is contained to the process |
| GIL impact (CPython) | serialized for Python bytecode | each process has its own GIL — true parallelism |
The famous race condition in the transactions lesson — the lost update — happens at the database level and the database's lock manager usually catches it. Here is the same bug at the thread level, in Python, with no safety net:
import threading counter = 0 def add_one(value): return value + 1 def increment(): global counter for _ in range(1_000_000): counter = add_one(counter) # read, add, write back: looks like one step. is three. t1 = threading.Thread(target=increment) t2 = threading.Thread(target=increment) t1.start(); t2.start() t1.join(); t2.join() print(counter) # expected: 2_000_000. actual: less, and different on every run.
Run this repeatedly and you'll get different numbers: 1,620,641, then 1,344,139, then 1,583,130 on three runs with CPython 3.13. The GIL prevents some interleavings but not all — more on that below. The source of the bug is that reading counter, adding one and writing it back are three separate steps, and the interpreter can switch threads between them. The shorter spelling counter += 1 compiles to the same three steps (LOAD, BINARY_OP, STORE). On current CPython (3.10 and later) that bare loop usually prints the right total, because the interpreter now switches threads only at the end of a loop pass or at a function call. That hides the bug; it does not remove it. Put any call between the read and the write, as add_one does here, and the lost updates are back.
At the bytecode level, counter += 1 is:
LOAD_GLOBAL counter— read the current value into a registerBINARY_OP +— add 1STORE_GLOBAL counter— write the new value back
If thread 1 reads 42, thread 2 reads 42, thread 1 stores 43, thread 2 stores 43 — one increment is lost. This is the lost update from Lesson 3.6, reborn without the database's protection.
SEE ITRace condition simulator
Now be the scheduler. At the TraceLab depot, clerks T1 and T2 each have to add one parcel to the count on the board, and each does counter += 1 the way a thread does, as three cards: read the board (LOAD), add one to its own copy (ADD), write the copy back (STORE). Nothing moves until you pick which clerk plays its next card. Round 1: make the two additions end with the board at 1. Then a key is added and you try again, and with two keys you try to freeze both clerks.
Notice what happened. In round 1 the board ended at 1 because both clerks read 0 before either wrote back: each added one to a copy that was already out of date, and the second write covered the first. That is the lost update, and it is the race in counter += 1: one line of source is three steps, and a switch between them is enough. Of the 20 orders in which two clerks can play three cards each, 18 lose a count; only the two that never switch part-way are safe. A real scheduler switches far less often than you did, which is why the bug hides until the load is high.
In round 2 the key was a lock. The clerk that took it played read, add one and write back with nobody in between, so the marked floor in front of the board, the critical section, held one clerk at a time and every order ended at 2. In round 3 each clerk took one key and waited for the other: a deadlock. Each held what the other needed, you had no card left to play, and the operating system does not step in either; the program hangs. When both clerks took the round key first, you could not build the freeze, because a clerk that waits holds nothing. That rule is lock ordering, and the part on locks and deadlocks below comes back to it.
CORE CONCEPTThe GIL: what it does and what it doesn't
Every Python engineer has a half-baked understanding of the GIL. Here is the precise version.
CPython — the reference implementation — has a Global Interpreter Lock: a mutex that must be held to execute Python bytecode. One thread runs at a time. The GIL exists because CPython's memory management, specifically reference counting, is not thread-safe. Without the GIL, two threads could increment or decrement a reference count concurrently and corrupt the garbage collector. The GIL is the lazy but effective solution: serialize all bytecode execution, problem solved.
What this means in practice: in CPython, Python threads cannot execute Python bytecode in parallel on multiple cores. This makes threads useless for CPU-bound tasks — you won't get a speedup from spinning up 8 threads to do numeric work. The GIL turns them serial.
The most dangerous misconception: "I'm using Python so the GIL protects me from race conditions." The GIL serializes bytecode execution between thread switches, but a thread switch can happen between any two bytecodes. The three-bytecode sequence LOAD/ADD/STORE is not atomic. The race on a shared counter is real in CPython, as the game shows, and race.py loses updates on every run. A bare counter += 1 loop that happens to come out right on a current version is an accident of where the interpreter switches threads, not a guarantee. The GIL is not a substitute for proper locking.
What the GIL does help with: operations that are a single bytecode instruction tend to be atomic in practice — x = 42 (STORE_NAME), appending to a list (LIST_APPEND). This is an implementation detail that can change between CPython versions, and code that relies on it is fragile. Write correct synchronized code regardless.
The GIL is released during I/O — file reads, network calls, database queries, time.sleep(). While thread A waits for the network, thread B can execute Python bytecode. This is why threads are useful for I/O-bound concurrency in Python: while one thread blocks waiting for an HTTP response, other threads can run. Lesson 3.8 covers the async alternative, which is more efficient for the same problem.
CPython 3.13 introduced experimental support for a "no-GIL" build (PEP 703), which disables the GIL and uses fine-grained locking instead. It is not yet the default, and it shifts the burden of correctness from "CPython does it for you (partly)" to "you must write correct concurrent code." Nothing about the races you learned above changes — they become more likely to manifest, not less.
IN PRACTICEMultiprocessing for CPU-bound work
If threads can't parallelize CPU-bound Python code (because the GIL serializes execution), the answer is multiprocessing: spawn multiple processes, each with its own Python interpreter, its own GIL, and its own memory space. True parallelism, at the cost of isolation — no shared memory means you have to explicitly communicate.
from multiprocessing import Pool import time def heavy_work(n): # pure Python arithmetic — the GIL would serialize threads here result = 0 for i in range(n): result += i * i return result if __name__ == "__main__": # required on Windows/macOS (spawn) tasks = [10_000_000] * 4 start = time.monotonic() with Pool(4) as p: results = p.map(heavy_work, tasks) elapsed = time.monotonic() - start print(f"4 processes: {elapsed:.2f}s, results: {results}")
On a 4-core machine this runs roughly 4× faster than one process, because each subprocess has its own GIL and runs genuinely in parallel. The cost: each process gets a full copy of the Python interpreter and your program's memory (on Linux, copy-on-write via fork; macOS changed its default to spawn in Python 3.8 due to Objective-C runtime safety concerns; Windows always uses spawn; check multiprocessing.get_start_method() on your platform). Passing data between processes requires serialization — Python uses pickle by default. Large NumPy arrays passed through a Pool will be pickled and un-pickled, which can negate the speedup. Use shared memory (multiprocessing.shared_memory, added in Python 3.8) or memory-mapped files for large data.
The decision tree is short: I/O-bound and Python? → threads or async (Lesson 3.8). CPU-bound and Python? → multiprocessing or move the hot loop to C/NumPy/Cython. Mixed? → multiprocessing for the CPU parts, async or threads for I/O. The GIL is a Python-bytecode serializer; C extensions like NumPy release it during their own computation, so a NumPy matrix multiply can run in parallel across Python threads — the GIL is released for the duration of the C call.
Locks and deadlocks
Back to threads. The standard fix for a race condition is a lock (also called a mutex — mutual exclusion). A lock has two operations: acquire (block until you hold it) and release (give it up). The rule: always acquire the lock before entering a critical section; always release it after. Python's threading.Lock() and the with statement make the release guaranteed even on exceptions:
lock = threading.Lock() counter = 0 def safe_increment(): global counter for _ in range(100_000): with lock: # acquire on entry, release on exit (even on exception) counter += 1 # now truly atomic from other threads' perspective
This gives the correct result — 200,000 — every time. But it introduces a new problem: deadlock. Deadlock is the situation where two (or more) threads are each waiting for a lock held by the other, and neither can proceed. It requires four conditions to hold simultaneously (Coffman conditions, 1971): mutual exclusion, hold-and-wait, no preemption, circular wait. The standard prevention is lock ordering: always acquire locks in the same global order everywhere in the program. If every thread acquires lock A before lock B, the circular wait condition cannot arise.
lockA = threading.Lock() lockB = threading.Lock() def thread1_work(): with lockA: # acquires A time.sleep(0.001) # context switch happens here in practice with lockB: # waits for B — but thread2 holds B, waiting for A do_work() def thread2_work(): with lockB: # acquires B with lockA: # waits for A — deadlock do_work()
TRACELABTraceLab v2: concurrent writers and the genuine race
The Stage 3 milestone is TraceLab v2: a working client–server system with an HTTP API, Postgres backend, and token auth. In a robotics lab, multiple robots write telemetry concurrently — and this is where we deliberately expose the race that the v0 CSV logger glossed over.
In v0, the "no locking" debt was listed: "two robots log at once; rows shear mid-line." In v2, we have replaced the CSV with Postgres and solved the file-write race, but a subtler race appears at the application layer: the read-modify-write pattern for run counters.
# TraceLab v2 — two concurrent HTTP requests can both reach this handler # at the same instant. Without proper isolation, the run counter races. async def create_run(project_id: int, db): # BAD: read-modify-write without a transaction row = await db.fetchone( "SELECT run_count FROM projects WHERE id = $1", project_id ) new_count = row["run_count"] + 1 # both requests read 41 await db.execute( "UPDATE projects SET run_count = $1 WHERE id = $2", new_count, project_id # both write 42 — one increment lost ) return new_count # both return "run 42" — now you have duplicate run IDs # GOOD: let the database serialize it with an atomic UPDATE async def create_run_safe(project_id: int, db): row = await db.fetchone( """UPDATE projects SET run_count = run_count + 1 WHERE id = $1 RETURNING run_count""", project_id ) return row["run_count"] # the DB guarantees uniqueness — Lesson 3.6 pays off
The fix is exactly what Lesson 3.6 taught: push the read-modify-write into a single SQL statement so the database's row-level lock makes it atomic. At the thread/process level, the protection is the database. The lesson is that concurrency bugs are not just about shared in-memory state — they appear at every layer where state is mutated and multiple actors can arrive simultaneously.
Don't add concurrency until something measurable demands it. "My training loop might be slow" is not that thing. Concurrency adds a class of bugs that is qualitatively harder to test, debug, and reason about. The first optimization is always algorithmic; the second is usually I/O restructuring; the third might be parallelism. Each step is cheaper than the next. Get a working, correct sequential version first — then measure.
The v2 codebase is intentionally left with what the brief calls the "God module" problem — a single large file with intertwined HTTP handling, database logic, and auth. That debt is Stage 4's raw material: we'll decompose it with proper layering, interfaces, and dependency injection. For now, it works, and it's ours to dissect.
WEAKNESSWhere concurrency bites
The GIL prevents truly simultaneous bytecode execution, but thread switches happen between any two bytecodes. Multi-step operations like read-modify-write are not atomic. Engineers who believe "Python is safe because of the GIL" have been bitten by counter corruption, list mutation bugs, and dictionary resize races that the GIL did not prevent. The rule: if you're mutating shared state from multiple threads, use a lock. Always. Even in Python.
A common trap: "I have 8 cores, I'll use 8 threads to parallelize my training loop." With the GIL, Python bytecode is serialized — you won't get 8× speedup, you'll likely get worse performance than 1 thread (lock contention overhead). Profile first. If your bottleneck is Python bytecode (not I/O, not a C extension), use multiprocessing, not threading. If your bottleneck is NumPy/PyTorch operations, they already release the GIL — threads might help there, but measure.
Concurrent code is not just harder to write — it's qualitatively harder to test. Race conditions often only appear under specific timing conditions that unit tests don't reproduce. Once you add threads to a codebase, every test that touches shared state needs to think about whether it's testing under realistic concurrency. The cost of correctness rises steeply. The senior engineer's reflex is not "how do I add threads?" but "can I redesign this so threads aren't necessary?" — immutable data, message passing, and proper use of databases for shared state eliminate most threading needs before they arise.
KEY INSIGHTHow this connects
- Databases III: Transactions and Isolation (3.6) — the lost update you studied there is the same hazard you met here. Isolation levels are the database's answer to the race conditions threads create; they're conceptually identical problems in different contexts. When you push your critical section into an atomic SQL
UPDATE … RETURNING, you're borrowing the database's correctness guarantee. - The Operating System Contract (1.3) — threads and processes are OS primitives. The scheduler that creates interleavings, the kernel that enforces process isolation, the syscall boundary — all of this was introduced there. Concurrency makes those abstractions load-bearing: a bug in your threading model can't be debugged without understanding what the scheduler does.
- Concurrency II: Async and the Event Loop (3.8) — next lesson, you'll see the alternative to threads for I/O-bound work: a single thread with cooperative scheduling (async/await). No shared mutable state, no races, no locks — but a different failure mode (blocking the event loop). The comparison makes both models sharper.
GLOSSARYKey terms
- Thread
- The smallest OS-schedulable unit of execution. Threads within a process share the same heap and address space, enabling fast communication but creating race condition hazards.
- Process
- An OS-managed running program with its own private address space. Processes are isolated from each other; communication requires explicit IPC mechanisms.
- Race condition
- A bug where program correctness depends on the uncontrolled order or timing of concurrent events. Manifests as non-deterministic behavior that's notoriously hard to reproduce and debug.
- Critical section
- A region of code that accesses shared mutable state and must not be executed by more than one thread at a time. Protected by locks.
- Mutex (lock)
- A synchronization primitive with acquire/release semantics. Only one thread holds it at a time; others block on acquire until it's released. Prevents races; can cause deadlocks if misused.
- Deadlock
- A state where two or more threads are each waiting for a resource held by another, forming a circular wait. No thread can proceed. Prevention: always acquire locks in the same global order.
- GIL (Global Interpreter Lock)
- CPython's per-interpreter mutex that serializes Python bytecode execution. Prevents true CPU-bound parallelism with threads; does not prevent all race conditions.
- Concurrency vs parallelism
- Concurrency is a structural property (tasks that could overlap); parallelism is an execution property (tasks that literally run simultaneously on different hardware). Concurrency can exist on one core; parallelism requires multiple.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1Why does counter += 1 produce a race condition even though it looks like one operation?›
At the bytecode level it compiles to three instructions: LOAD the current value, ADD 1, STORE the new value. The thread scheduler can preempt between any two bytecodes. If two threads both LOAD the same value (say, 42), then both ADD to get 43, and both STORE 43 — one increment is lost. Source-code atomicity does not imply bytecode atomicity. The critical section spans three instructions that must execute without interruption from the other thread.
2The GIL serializes Python bytecode execution. Does this prevent race conditions? Why or why not?›
No. The GIL prevents two Python bytecodes from executing simultaneously — but thread switches happen between any two bytecodes. A critical section that spans multiple bytecodes (like LOAD/ADD/STORE) is not protected by the GIL. The GIL protects CPython's internals (reference counting) from concurrent corruption; it does not protect your application-level shared state. You must use explicit locks for correct concurrent mutation.
3When should you use multiprocessing instead of threading in Python? What's the tradeoff?›
Use multiprocessing for CPU-bound work — tasks where the bottleneck is Python bytecode computation. Threads can't parallelize this because the GIL serializes execution. Multiprocessing spawns separate interpreter instances, each with its own GIL, enabling true CPU parallelism. The tradeoff: no shared memory by default, so communicating data between processes requires explicit serialization (pickle) with associated overhead. Large data (like NumPy arrays) should use shared memory or memory-mapped files to avoid copy costs. Threads remain appropriate for I/O-bound work where the GIL is released during blocking calls.
4What is a deadlock, how does it arise, and what is the standard prevention strategy?›
A deadlock is a state where two or more threads are each waiting for a lock held by the other, so none can proceed. It arises when: thread A holds lock 1 and wants lock 2; thread B holds lock 2 and wants lock 1 — circular wait. The standard prevention is lock ordering: define a global total order on locks and always acquire them in that order everywhere in the program. If every thread always acquires lock A before lock B, the circular wait condition can't arise. Python's threading.RLock (reentrant lock) helps when a single thread needs to acquire the same lock twice, but doesn't solve multi-lock circular waits.