Memory: The Machine Beneath the Variables
Why b = a copies nothing, why your training script eats 60 GB by epoch 40, and why "pointer bugs" exist in Python too — they're just called aliasing.
🎯 By the end you'll understand
How Python names, objects, and references actually work; why assignment never copies; what reference counting is and what it costs; why a one-million-row list of dicts can eat gigabytes; when Python's garbage collector runs and why it sometimes doesn't; and how three classic aliasing traps — mutable default arguments, shared nested lists, and is vs == — corrupt experiments silently.
KEY INSIGHTPython has no variables — only names and objects
When you write b = a, you do not copy anything. You give the same heap object a second name. Both names point at one object, and mutations through either name affect both.
You have run hundreds of Python experiments. Each one created variables, assigned values, passed lists to functions, appended metrics. Every one of those operations looked like value manipulation. Almost none of it was. Python has no variables in the C sense — no box that contains a value. It has names that point to objects, and objects that live on the heap. The instant you internalize this, a whole category of bugs stops being mysterious and starts being predictable.
In Python, every assignment is the binding of a name to an object, never a copy of the object. The right-hand side is always evaluated to a heap object; the left-hand side is a name that is pointed at that object. b = a makes b and a aliases — two names for one thing. Mutation through either name mutates the single shared object. This is not a Python flaw; it is a deliberate design that makes large data structures cheap to pass around. But it will bite you the first time you assume otherwise.
CORE CONCEPTNames, objects, and the binding model
Python's data model draws a clean line between two things that most languages conflate:
- Objects — live on the heap; have an identity (address), a type, and a value. They are reference-counted and eventually collected.
- Names — live in a namespace (a dict, more or less). A name is just a string that maps to an object reference. Assigning a name changes the mapping; it does nothing to the object.
Work through this sequence carefully:
a = [1, 2, 3] # ONE list object, born on the heap; 'a' points to it b = a # 'b' now points to the SAME object. NO copy was made. b.append(4) # mutates the single object through the name 'b' print(a) # [1, 2, 3, 4] ← surprise if you expected a copy # rebinding 'b' to a new object does NOT affect 'a' b = [10, 20] # b now points at a brand-new object print(a) # [1, 2, 3, 4] ← unchanged
The key distinction: b.append(4) is a mutation through the existing reference — the object changes in place. b = [10, 20] is a rebinding — the name b is redirected to a different object, leaving the original untouched. Every piece of confusing Python behavior around assignment is one of these two operations, or a confusion between them.
You can verify the identity directly. id(obj) returns the object's memory address in CPython. Before rebinding, id(a) == id(b) is True. After rebinding, they diverge.
Integers, floats, strings, and tuples are immutable — once created, they cannot be mutated in place. So aliasing them is harmless: there is no operation that changes a str object through one of its names and surprises you through another. The danger is exclusively with mutable objects: lists, dicts, sets, and most user-defined objects. CPython even caches small integers (−5 to 256) and interned strings as a further optimization — which is why a = 1; b = 1; a is b is True but should not be relied upon. More on this in the widget's is vs == challenge.
b redirects its arrow without touching the object that a still points to.HOW IT WORKSStack frames, heap objects, and the call model
Lesson 1.1 located the heap inside the process; now we zoom in. In CPython, a function call creates a frame object — a heap-allocated struct that holds the local namespace (a dict of name→object references), the instruction pointer, and a link to the enclosing frame. This is why Python frames survive a return: they are ordinary objects, not C-stack slabs.
When you call log_run(values) in TraceLab:
A new frame object is allocated on the heap
Python allocates a PyFrameObject with room for the function's local variables. The parameter values gets an entry in that frame's namespace dict that points at the same list the caller passed — no copy.
Each name binding inside the function updates the frame dict
f = open("runs.csv", "a") allocates a new file object on the heap and binds f in the local namespace to it. The file object's refcount goes to 1.
On return, the frame is released
CPython decrements the refcount on every object in the frame's namespace. Objects whose refcount drops to zero are freed immediately. If f was the only reference to the file object, the file is closed and deallocated right here — this is the mechanism behind CPython's (fragile) RAII-like cleanup.
The implication for passing arguments is exact: Python always passes object references, never values. The C community calls this "pass by object reference" to distinguish it from both C's value semantics and true reference semantics. Mutating the object through the parameter mutates it everywhere. Rebinding the parameter to a new object affects only the local name.
def mutate(lst): lst.append(99) # mutates the shared object → caller sees the change def rebind(lst): lst = [99] # rebinds local name → caller sees nothing data = [1, 2, 3] mutate(data) print(data) # [1, 2, 3, 99] rebind(data) print(data) # [1, 2, 3, 99] ← unchanged
CORE CONCEPTReference counting: the real-time memory manager
Every Python object carries a reference count — an integer stored in the object header that says how many names (or containers, or argument slots) currently point at it. CPython maintains this count with obsessive precision:
- A new object starts with refcount 1 (the expression that created it holds one reference).
- Binding a name or storing into a container increments the count.
- A name going out of scope, a container element being replaced, or an explicit
deldecrements it. - When the count hits zero, CPython calls the object's destructor immediately — no waiting, no collection pass.
You can inspect refcounts: import sys; sys.getrefcount(obj) returns the count, but note it temporarily adds 1 (because passing obj as a function argument is itself a reference). The cost of this machinery is real: every push and pop of the value stack adjusts refcounts; every function call and return adjusts dozens. It is one of the reasons the CPython interpreter pays more per opcode than a naive analysis would suggest.
Why refcounting at all? Because it provides deterministic, immediate cleanup. When a file object, a database connection, or a C extension's allocated buffer hits refcount zero, the destructor runs immediately and releases the resource. Tracing collectors (like Java's) don't promise this — you need explicit try/finally or context managers. CPython gives it to you for free in the common single-reference case. The catch: cycles.
The Global Interpreter Lock exists largely to protect reference count operations. Without the GIL, two threads could simultaneously increment and decrement the same object's refcount, producing a race condition that corrupts or leaks memory. Changing refcount increments to use atomic instructions (what Python 3.13's "free-threaded" mode does) is expensive — each atomic operation is ~7× slower than a plain integer add on modern CPUs. The GIL was the cheap solution. Free-threading (PEP 703, CPython 3.13) trades GIL removal for per-object locks and some performance regression on refcount-heavy workloads.
SEE ITMemory diagram simulator
Execute Python statements one at a time and watch the heap, frame namespaces, reference arrows, and refcounts update live. Then face three aliasing-trap challenges: predict the outcome before stepping through to see if you were right.
The three challenges represent the three most common aliasing bugs in research code. The mutable default argument is a Python design quirk that bites nearly every new Python programmer and many experienced ones. The shared nested list is the exact bug that can corrupt your metric history matrix. And is vs == is a correctness trap that occasionally passes tests due to CPython's integer cache, making it especially pernicious.
IN PRACTICEThe real cost of Python objects
Now we can answer the opening question: why does a one-million-row list of dicts eat gigabytes? The answer lives in Python's object model.
Every Python object carries a mandatory header: a refcount (8 bytes on 64-bit), a type pointer (8 bytes), and type-specific fields. A Python int is not a 4-byte integer — it is a 28-byte heap object containing a refcount, type pointer, and the integer's value. A Python float is similarly 24 bytes. Verify right now in your REPL:
import sys print(sys.getsizeof(1)) # 28 bytes (int) print(sys.getsizeof(1.0)) # 24 bytes (float) print(sys.getsizeof("hello")) # 54 bytes (str: header + 5 chars) print(sys.getsizeof({})) # 64 bytes (dict, empty) print(sys.getsizeof([])) # 56 bytes (list, empty)
Now imagine 1 million experiment records stored as dicts, one dict per row, each with 10 float metrics. Each dict is 64 bytes minimum, plus 8 bytes per slot for the hash table, plus 24 bytes per float value, plus the interned string keys. A rough lower bound: 1M × (64 + 10×8 + 10×24) = 1M × 384 bytes = 384 MB. For data that would fit in 80 MB as raw 64-bit floats in a NumPy array. The multiplicative overhead is real, and at 10M rows it becomes the memory budget.
| Storage format | 1M rows × 10 floats (approx) | Notes |
|---|---|---|
| Python list of dicts | ~400–600 MB | Per-object overhead, hash tables, pointer indirection |
| Python list of lists | ~180–220 MB | No dict overhead, but still boxed floats |
| NumPy 2D array (float64) | ~80 MB | Unboxed; one contiguous C array; no per-element Python objects |
| Pandas DataFrame (float64) | ~80 MB + frame overhead | Backed by NumPy blocks; metadata is a small fixed cost |
This is the concrete version of "Python is slow for data": it is not primarily about execution speed, it is about memory layout. NumPy packs floats into a flat C array with no per-element Python objects. CPython's VM never touches the individual floats during vectorized operations — it calls into compiled C code that processes the array as raw memory. The 5–10× memory reduction comes for free, and cache locality makes the computation faster too.
A common pattern in research: accumulate per-step metrics as Python objects in a growing list across an epoch, never clear the list between epochs, and repeat for 40 epochs. The list holds references to tens of millions of boxed Python objects, none of which can be freed because the list still points to them. The refcounts never reach zero. The fix is almost always to: (a) convert to NumPy or a pre-allocated array immediately, (b) clear the accumulator per epoch, or (c) write to disk and not keep metrics in memory at all. This is exactly TraceLab's memory model problem, described precisely.
HOW IT WORKSGarbage collection: when refcounting isn't enough
Reference counting has one structural weakness: reference cycles. If object A holds a reference to object B, and B holds one back to A, and nothing else points to either of them, both refcounts are 1 — but neither will ever reach zero. CPython's primary refcount mechanism would leak them forever.
a = {}
b = {}
a['partner'] = b # a references b
b['partner'] = a # b references a — cycle!
del a
del b
# refcount of both objects is now 1 (from the cycle). They will NOT
# be freed by refcounting alone — the cyclic GC will find them later.
Python's solution is a supplementary cyclic garbage collector (module gc) that runs periodically. It works by generational tracking: newly created objects go into Generation 0; objects that survive a collection are promoted to Generation 1, then Generation 2. The collector runs when the count of new allocations since the last run exceeds a threshold (default: 700 for gen 0). A full gen-2 collection walks all tracked objects looking for cycles.
Practical consequences:
- Finalizers (
__del__) may not run promptly for cyclic objects. If your object's__del__closes a network connection, don't rely on it — usewithstatements (context managers) instead. - The GC can cause latency spikes. A large Python application that creates many short-lived objects can trigger gen-2 collection at unpredictable times, pausing execution. This is one motivation for using
gc.disable()in latency-sensitive hot paths and callinggc.collect()explicitly at safe points. - Most Python objects are not cyclic. Plain data objects (ints, strings, tuples, frozen data) are not tracked by the GC at all. The overhead is concentrated in dicts, lists, and user-defined objects.
with open(...) is not optionalIn CPython, f = open("runs.csv", "a") creates a file object. When f's refcount hits zero (because the function returns or you del f), CPython immediately calls the file's destructor, which flushes and closes the OS file descriptor. This appears to work reliably in CPython. In PyPy, Jython, or any future Python runtime without refcounting, there is no such guarantee — the file might stay open for seconds or forever. The with statement inserts an explicit close() call at the end of the block regardless of implementation. This is why with open(...) as f: is not style — it is correctness.
TRACELABTraceLab v0 and its aliasing debt
Look at log_run in TraceLab v0 through this lesson's lens:
def log_run(values): f = open("runs.csv", "a") # file object born, refcount = 1 row = time.strftime("%Y-%m-%d %H:%M") for v in values: row = row + "," + v # ← NEW str allocated per iteration # old 'row' refcount drops to 0 → freed f.write(row + "\n") f.close() # ← redundant in CPython; mandatory elsewhere
Three memory observations worth logging:
- String concatenation in the loop allocates N new string objects, where N is the number of values. Each iteration creates a fresh string, hands refcount to the name
row, and immediately decrements the previousrowstring's count to zero, freeing it. This is correct behavior, but it means the loop is O(n) in allocations and O(n²) in total bytes copied — because each concatenation must copy all previous characters plus one new field. For 50 metrics per run, this is trivially fast. For 50,000 it is not. - The
valuesparameter is an alias for the caller's list. If you pass a list and the function mutated it (it doesn't, but imagine it did), the caller's list changes. This is the pass-by-object-reference model, live. f.close()is an explicit destructor call. In CPython the file would close on function return whenf's refcount hits zero. But we have just seen that is implementation-dependent. The correct fix — introduced when we improve v0 in Stage 2 — iswith open(...) as f:.
The deeper aliasing debt in TraceLab will appear when we add a results cache. A naive cache dict that stores references to metric lists will prevent those lists from being freed — the same memory-leak pattern described above. Recognizing it as a refcount issue is half the fix.
TraceLab v0 as written today doesn't exhibit aliasing bugs — it creates strings and immediately uses them. But the moment you add a feature like last_run = values to cache the most recent run for quick inspection, you have bound a module-level name to the caller's list. The next caller who passes the same list object and mutates it between calls will corrupt your cache. This is not hypothetical; it is the shape of most real Python caching bugs.
WEAKNESSWhere this mental model bites
The canonical Python beginner trap. A default argument value is evaluated once, when the def statement executes, not on each call. So def f(history=[]): creates exactly one list object that persists for the life of the function. Every call that doesn't pass an explicit argument shares that same list and mutates it. The idiomatic fix is the sentinel pattern: def f(history=None): if history is None: history = []. This creates a fresh list on each call. This bug is so common it appears in CPython's own source history and in published machine-learning library codebases.
b = a[:] and b = list(a) create a new list object whose slots contain the same object references as the original. If the list elements are mutable (nested lists, dicts), you still have aliasing inside the copy. To fully decouple nested structure you need copy.deepcopy(a) — which recursively copies every object in the graph. Deep copy is expensive and, for large structures with cycles, slow. The design question "do I need independence or just safety against top-level rebinding?" is worth asking explicitly before reaching for deepcopy.
is to test equalityis tests object identity — same heap address. == tests value equality. For small integers (−5 to 256) and interned strings, CPython caches singleton objects, so a is b may return True purely by coincidence of implementation. For integers above 256, two variables holding the same numeric value are almost certainly different objects. Code like if status is "ok": passes tests on small strings because CPython interns them — and silently breaks when the string is assembled dynamically and is no longer the interned instance. Use == for value comparison, full stop. Reserve is for checking against known singletons: if x is None:, if x is True:.
KEY INSIGHTHow this connects
- From Source to Execution (1.1) — we located the heap inside the process and promised Lesson 1.2 would map it object by object. This lesson delivers that map: every Python object you have ever created lived in that heap region, and
id(obj)is its address. - The OS Contract (1.3) — the process model and virtual address space that houses the heap becomes the full focus. Memory allocation itself is a negotiation between the allocator (CPython's
pymalloc), the C runtime'smalloc, and the kernel'smmap.
The mental model from this lesson — every assignment is a reference binding, never a copy — recurs in almost every later topic. Understanding aliasing is a prerequisite for: reasoning about shared mutable state in concurrent code (Stage 3), understanding how ORM objects share state with their session (Stage 4), and debugging why a cached API response "spontaneously" changes (Stage 5). The concepts are identical; the setting changes.
GLOSSARYKey terms introduced
- Name binding
- The operation performed by assignment: making a name in a namespace point to a particular object. It never copies the object.
- Alias
- Two or more names that refer to the same heap object. Mutation through any alias is visible through all others.
- Reference count
- An integer in each Python object's header tracking how many names or containers currently point to it. When it reaches zero, the object is freed immediately in CPython.
- Pass by object reference
- Python's argument-passing model: the parameter in a function receives a reference to the same object as the caller's variable — no copy. Mutation through the parameter is visible to the caller; rebinding the parameter is not.
- Shallow copy
- A new container whose slots are copies of the original container's references. The contained objects themselves are not copied, so nested mutable objects are still shared.
- Cyclic garbage collector
- Python's supplementary GC (module
gc) that detects and frees reference cycles — objects that are unreachable but whose refcounts are nonzero because they point at each other. - Mutable default argument
- A Python footgun: a default argument value is created once when the
defis executed, so a mutable default (list, dict) is shared across all calls that don't supply that argument. - Object identity vs equality
istests whether two names point to the exact same object (same address).==tests whether two objects compare as equal by value. These diverge for all objects except cached singletons.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1What does b = a actually do when a is a list? What does it not do?›
It makes the name b point to the same list object that a points to. It does not create a new list. After this assignment, a is b is True, a's refcount becomes 2, and any mutation through b (like b.append(x)) is visible through a. To get an independent copy of the list's top level, you need b = a[:] or b = list(a).
2Why can a Python float use 24 bytes when the number itself is only 8 bytes?›
Every Python object carries a mandatory header: an 8-byte reference count and an 8-byte type pointer. That is 16 bytes of overhead before the 8 bytes for the actual 64-bit float value. Total: 24 bytes. This overhead is why a list of one million Python floats uses far more memory than the equivalent NumPy array — NumPy stores raw unboxed 8-byte values with no per-element Python object overhead.
3Why does CPython not need a garbage collector for most objects — and when does it need one?›
Most objects are freed immediately when their reference count reaches zero, without any GC pass. CPython's cyclic garbage collector is needed only for reference cycles — objects that point at each other in a cycle, keeping each other's refcount above zero even when nothing else in the program can reach them. The GC periodically walks "tracked" objects (containers) looking for unreachable cycles and frees them.
4What is wrong with this function signature, and how do you fix it?
def record(name, tags=[]):›
The default value [] is evaluated once, when the def executes. Every call that omits tags shares the same list object, so tags.append(x) inside the function permanently adds to the shared list. Subsequent calls will see the accumulated contents of all previous calls. Fix: use a sentinel.def record(name, tags=None): if tags is None: tags = []
This creates a fresh list on each call that omits the argument.