Object Tracking: Kalman + Hungarian and the ID-Switch Problem
The ID-switch failure — when your tracker assigns the wrong ID to a returning person — is not an edge case in a crowd; it is the dominant failure mode of centroid-based trackers.
🎯 By the end you'll understand
Why detection and tracking are architecturally different problems that must be solved in sequence. How SORT (Simple Online and Realtime Tracking) combines a bounding-box Kalman filter with the Hungarian assignment algorithm. The mechanics of track birth, track death, and the gate threshold that controls both. Why ID-switches happen at occlusion boundaries — not during occlusion, but after it ends — and the role of appearance features in suppressing them. How FollowBot's single-target tracker is a simplified case of multi-object tracking and why it is still susceptible to ID-switch in crowded environments.
KEY INSIGHTAn ID-switch does not announce itself — it just starts following the wrong person
When a tracker re-identifies person B as person A after an occlusion, it does not log a warning. It does not lower its confidence score. It simply continues reporting positions — now for the wrong person — with exactly the same authority it had before.
Every tracking paper in the multi-object tracking (MOT) benchmark focuses on MOTA, HOTA, and ID-F1 scores. These aggregate across sequences. They don't tell you what happens at minute 3 in a real-world corridor when two people cross paths and one walks through a doorway. At that moment, centroid-based trackers built on IoU alone reliably fail — and the failure mode is not a crash. It is continued, confident, silent operation on a mis-identified target.
The ID-switch failure class belongs to the Estimate node of the sense-think-act loop — it is a state estimation failure, not a detection failure. The detector sees people correctly. The tracker assigns the wrong identity to a correctly-detected person. Everything downstream — the planner, the controller, the behavior tree — operates on a coherent but wrong identity. This is precisely what makes it dangerous: there is no observable anomaly except that FollowBot is following the wrong person.
CORE CONCEPTDetection and tracking are different problems — and they compose
A detector answers: "where are objects in this frame?" It produces a set of bounding boxes and class scores. Each call to the detector is independent; it has no memory of previous frames. YOLOv8 running on your RGB frame is a detector. The LIDAR cluster pipeline from Lesson 5.5 is a detector. Both produce per-frame observations with no temporal continuity.
A tracker answers: "which detection in this frame corresponds to which detection from the last frame — and the frame before that — and is the same physical object that has been assigned track ID 42 since it first appeared?" Tracking imposes temporal identity on a sequence of independent detections. It requires a motion model (to predict where each tracked object is going) and an assignment algorithm (to match new detections to existing tracks).
The standard architecture is detect-then-track:
The key architectural consequence: if the detector drops a frame — because the person is occluded, the light changes, the confidence threshold isn't met — the tracker doesn't know about the detection gap. It only sees "no detection matched this track." How it handles that gap — how long it waits before declaring the track dead, whether it re-identifies the target when it reappears — determines whether you get an ID-switch.
HOW IT WORKSSORT: four components, one pipeline per frame
SORT (Bewley et al., 2016) is the canonical minimal multi-object tracker. It is deliberately simple: it uses only bounding-box geometry, no appearance features. Understanding SORT precisely — including why it fails — is the prerequisite for understanding every more sophisticated tracker that follows.
Kalman filter state for a bounding box
Each track maintains a Kalman filter with an 8-dimensional state vector: [u, v, s, r, u̇, v̇, ṡ, ṙ] where (u, v) is the center of the bounding box, s is the scale (area), r is the aspect ratio, and the dotted quantities are their first derivatives (velocities). The observation vector is 4-dimensional: [u, v, s, r] — the raw detection. The motion model is constant-velocity: x_{t+1} = F·x_t + noise where F is the identity-with-velocity matrix. Aspect ratio r is assumed constant — a reasonable approximation for a walking person tracked from one camera angle. This is a linear Kalman filter, not an EKF, because the bounding-box motion model is linear. (You built the EKF for the nonlinear case in Lesson 4.5; SORT doesn't need it.)
Predict: propagate all tracks forward one timestep
Before looking at any new detections, every existing track's Kalman filter runs its prediction step: x̂ = F·x, P̂ = F·P·Fᵀ + Q. This produces a predicted bounding box position for every track. These predictions are where the tracker expects the objects to be if they kept moving at their current estimated velocity. The prediction step runs even if the object was not detected in the previous frame — the filter propagates the prior forward using the motion model alone, widening the covariance ellipse. This is the same predict-then-update loop you saw in the EKF; SORT is an application of the Kalman recipe to bounding boxes.
Assign: Hungarian algorithm on IoU cost matrix
SORT builds a cost matrix where C[i][j] = 1 - IoU(predicted_box_i, detection_j) and IoU is the intersection-over-union between the predicted bounding box of track i and the new detection j. Then it runs the Hungarian algorithm (also called the Kuhn-Munkres algorithm) to find the minimum-cost one-to-one assignment. The Hungarian algorithm solves the assignment problem in O(n³) time — for n tracks matching n detections. An IoU gate is applied: if the best-matching IoU between a track and any detection is below a threshold (typically 0.3), the track is not matched — instead it is incremented as "missed." This gating prevents distant detections from being matched to tracks they cannot plausibly belong to.
After assignment: matched detections are used to update their matched track's Kalman filter. Unmatched detections become candidate new tracks. Unmatched tracks are aged: after max_age consecutive missed frames (typically 1–3), the track is deleted.
Update: correct each matched track's Kalman filter
For each matched (track, detection) pair, run the Kalman correction step: K = P̂·Hᵀ·(H·P̂·Hᵀ + R)⁻¹ (Kalman gain), then x = x̂ + K·(z - H·x̂) (state update) and P = (I - K·H)·P̂ (covariance update). H is the observation matrix that extracts [u, v, s, r] from the 8D state. The result: the track's position estimate is pulled toward the new measurement, weighted by the relative confidence of the motion model (process noise Q) vs. the detector (observation noise R). Track birth: a new track is confirmed after it receives detections in at least min_hits consecutive frames (typically 3), to suppress spurious single-frame detections.
IoU works as an assignment signal when two conditions hold: (1) the frame rate is high enough that objects don't move more than their own size between frames, and (2) objects don't occlude each other. Under 30 Hz video and walking-speed pedestrians, condition (1) usually holds. Condition (2) fails reliably in crowds. When two tracks predicted boxes overlap in the cost matrix — because the objects are crossing — the Hungarian assignment is operating on ambiguous geometry. At that moment, which track gets which detection is determined by which predicted box happens to have a slightly higher IoU with each detection. That is the precise origin of the ID-switch.
WEAKNESSThe ID-switch problem: mechanics and causes
An ID-switch (also called an identity switch, or IDSW in MOT benchmarks) occurs when the tracker assigns a detection of person A to track B (or vice versa), causing the two tracks to swap their identities. The ID counter continues incrementing — the tracker never resets, it just mis-assigns. Here is the precise failure sequence:
Three conditions jointly produce an ID-switch in SORT:
- Spatial ambiguity at re-appearance. When A reappears, its new detection's IoU with track B's prediction is higher than with track A's prediction. This happens when A was occluded long enough for track A's prediction to drift, or when A reappeared near where B currently is.
- No appearance signal. SORT uses only geometry (IoU). If A and B look different, an appearance-aware tracker would correctly re-identify A despite the geometric ambiguity. SORT cannot.
- Track A survived the occlusion. If track A was killed (max_age expired) during the occlusion, then when A reappears it gets a fresh ID — not a switch back to A, but a new ID (say, ID: C). That is a track fragmentation, not an ID-switch. Both are bad; they just break in different ways.
Once SORT makes an ID-switch, it cannot undo it. There is no consistency check, no global re-optimization. The swapped assignment propagates forward indefinitely. For FollowBot, this means: if FollowBot is tracking person A (the target), and an ID-switch occurs that transfers A's track ID to person B, FollowBot will follow person B from that point forward, with full confidence, until person B leaves the scene or another occlusion event occurs. There is no velocity discontinuity, no confidence drop, no ROS log warning. The only observable symptom is that the robot starts following the wrong person.
SEE ITMulti-object tracker simulator
This simulator runs a SORT-style tracker with 3 targets in a 2D arena. Step through time, trigger occlusion events, and watch ID-switches emerge from the predict-assign-update cycle. The cost matrix is shown at each timestep. Toggle the appearance model to see how re-identification suppresses switches.
Notice the asymmetry between the gate size effects. A very tight gate (low IoU threshold) causes tracks to lose their detections frequently — leading to track deaths and births with new IDs, fragmenting long tracks. A very loose gate (high IoU threshold) allows geometrically distant detections to be assigned, creating ID-switches even without occlusion. The "correct" gate size is problem-dependent and must be tuned for the specific camera frame rate, object motion speed, and density.
IN PRACTICESolutions and their tradeoffs
The tracking literature has produced several approaches to the ID-switch problem, each trading off complexity, latency, and compute:
| Approach | Core idea | Reduces ID-switch? | Cost | Used in |
|---|---|---|---|---|
| IoU gating only (SORT) | Assign by bounding-box overlap; discard if IoU < threshold | Partially — prevents false cross-frame matches at large gaps | Negligible | SORT baseline |
| Appearance re-ID (DeepSORT) | Combine IoU with a learned 128-D appearance descriptor; match on both | Significantly — appearance similarity survives short occlusions | ~5–15 ms per frame for descriptor extraction (GPU) | DeepSORT, ByteTrack |
| Larger max_age | Keep tracks alive for more missed frames before deleting | Avoids track fragmentation; can still switch if geometry is ambiguous | More "zombie" tracks consuming assignment capacity | Tunable in all trackers |
| Kalman velocity-gating | Gate not just on IoU but on Mahalanobis distance in state space (position + velocity) | Yes — two tracks crossing at different velocities are disambiguated | Requires consistent velocity estimates (needs dense frames) | SORT variants, StrongSORT |
| Single-target tracking (FollowBot simplification) | Only track one person; nearest-neighbour assignment | No ID-switch in isolation — but susceptible in crowds (switches to a different person permanently) | Minimal | Simple follow-me robots |
DeepSORT is the most widely deployed extension of SORT. It adds a CNN-based appearance descriptor (a 128-D embedding vector extracted from each detection crop) and uses a distance metric combining IoU and cosine distance in embedding space for the assignment cost. On short occlusions (less than ~2 seconds), the appearance descriptor is stable enough to correctly re-identify the person when they re-appear, even when the geometry is ambiguous. Longer occlusions degrade descriptor stability. The descriptor extraction runs as a separate model pass (~5–15 ms on GPU), making the total tracking pipeline about 20–30 ms per frame.
ByteTrack (Zhang et al., 2022) takes a different approach: instead of adding appearance features, it changes the assignment pipeline to use every detection — including low-confidence ones — in a two-stage assignment. High-confidence detections are assigned first (standard SORT logic); low-confidence detections are then assigned only to unmatched tracks, serving as re-identification anchors after occlusion. This recovers significantly more true positives without requiring an appearance network, at the cost of sensitivity to false positives. On crowded benchmarks ByteTrack often outperforms DeepSORT without the appearance GPU requirement — but its behavior in the specific person-following scenario (where you care about which specific person, not just detection count) is more nuanced.
When your tracker produces unexpected ID-switches or track losses, the first debugging step is to log and inspect the cost matrix at the moment of failure. The matrix shows exactly which tracks were competing for which detections, what the IoU values were, and which assignments the Hungarian algorithm made. A gate that is too tight shows up as an entire row or column of "gated" (high-cost) entries. A crossing-path ambiguity shows up as two rows with nearly equal costs for two columns. These patterns are diagnostic in a way that "the tracker lost the person" is not.
FOLLOWBOTSingle-target tracking and the crowd problem
FollowBot's person-tracking problem is simpler than the general multi-object tracking problem in one sense: you only care about one person. You don't need to maintain a gallery of N IDs; you just need to follow one target. This simplification is load-bearing for the system architecture.
In FollowBot's pipeline (built across Lessons 5.5–5.6), the tracking layer sits above the fusion output from Lesson 5.5: a 3D person position estimate at 10 Hz, produced by fusing LIDAR cluster centroids with YOLOv8 bounding boxes. The tracker receives this position stream and must:
Maintain a single-target Kalman filter in 3D (x, y, z + velocities)
The state vector is [x, y, z, vx, vy, vz] in the map frame. The measurement is the fused LIDAR-camera 3D position from Lesson 5.5. The motion model is constant-velocity (appropriate for a walking person over the 0.1s between 10 Hz updates). The filter is a linear KF — the map-frame 3D motion model is linear. Process noise Q is set to account for up to 1 m/s² acceleration changes (a person changing direction quickly). Observation noise R reflects the residual fusion error: roughly 0.05–0.15 m standard deviation in x and y at 3 m range, larger at longer range.
Nearest-neighbour assignment with a distance gate
Since there is only one track, the "Hungarian algorithm" degenerates to nearest-neighbour: find the detection closest to the predicted track position, accept it if within a gate radius (typically 1.0–1.5 m), reject it otherwise. If no detection is within the gate for more than max_miss consecutive frames (typically 10–15 frames at 10 Hz, i.e., 1–1.5 seconds), declare the track lost and trigger the recovery behavior (rotate-and-search).
Where ID-switch still occurs in single-target mode
Here is the failure that single-target simplification does not eliminate. If two people are in the scene, both detected by the LIDAR-camera pipeline, and the target person (A) walks behind an obstacle while a bystander (B) is nearby, then when A reappears, the gate may accept B's position as "close enough" to A's predicted position. The tracker assigns the measurement from B to track A. From this point forward, FollowBot follows B. This is structurally the same as an ID-switch in multi-object tracking — except that instead of swapping two stored IDs, you are switching the single tracked target from the desired person to a bystander. The fix requires the same tools as DeepSORT: an appearance descriptor or a semantic signal (e.g., the person is wearing a specific color, or a UWB tag) to disambiguate the re-identification.
At the current Stage 5 milestone, FollowBot's tracker is single-target nearest-neighbour with a 3D Kalman filter, no appearance model. This is intentional — it is the correct thing to build first. It works reliably in uncluttered environments and at the speeds (1.0–1.5 m/s walking) encountered in the test environment. The failure envelope is: crowds of more than 2–3 people in the tracker's sensor field, occlusions longer than about 1.5 seconds, and scenarios where a bystander crosses within 1.0 m of the predicted target position. Stage 6 will introduce the appearance model extension when FollowBot encounters more demanding environments.
The tracker node publishes a geometry_msgs/PoseWithCovarianceStamped on /followbot/target_pose at 10 Hz. The covariance matrix reflects the Kalman filter's current uncertainty — the behavior tree reads this and widens the approach corridor when uncertainty is high. Track-lost events trigger a ROS 2 action server call to the recovery behavior. This integrates with the SLAM Toolbox map (also Stage 5) so that the planner can execute the approach-person trajectory on the occupancy grid already built.
import rclpy, numpy as np from rclpy.node import Node from geometry_msgs.msg import PoseWithCovarianceStamped, Point from std_msgs.msg import Bool class PersonTracker(Node): def __init__(self): super().__init__('person_tracker') # Kalman state: [x, y, vx, vy] (2D map-frame, z treated separately) self.x = np.zeros(4) self.P = np.eye(4) * 1.0 # initial covariance dt = 0.1 # 10 Hz self.F = np.array([[1,0,dt,0], # constant-velocity model [0,1,0,dt], [0,0,1,0], [0,0,0,1]]) self.H = np.array([[1,0,0,0], # observe x, y [0,1,0,0]]) self.Q = np.eye(4) * 0.01 # process noise self.R = np.eye(2) * 0.04 # sensor noise (~0.2 m σ) self.gate = 1.5 # metres; reject if farther self.miss_count = 0 self.MAX_MISS = 15 # ~1.5 s at 10 Hz self.initialized = False self.create_subscription(Point, '/followbot/person_3d', self.meas_cb, 10) self.pub = self.create_publisher( PoseWithCovarianceStamped, '/followbot/target_pose', 10) self.lost_pub = self.create_publisher(Bool, '/followbot/target_lost', 10) self.create_timer(0.1, self.predict_cb) # 10 Hz predict def predict_cb(self): if not self.initialized: return self.x = self.F @ self.x self.P = self.F @ self.P @ self.F.T + self.Q self.miss_count += 1 if self.miss_count > self.MAX_MISS: self.lost_pub.publish(Bool(data=True)) def meas_cb(self, msg): z = np.array([msg.x, msg.y]) if not self.initialized: self.x[:2] = z; self.initialized = True; return pred_pos = self.H @ self.x if np.linalg.norm(z - pred_pos) > self.gate: return # outside gate — possible ID-switch; reject S = self.H @ self.P @ self.H.T + self.R K = self.P @ self.H.T @ np.linalg.inv(S) self.x = self.x + K @ (z - pred_pos) self.P = (np.eye(4) - K @ self.H) @ self.P self.miss_count = 0 self._publish() def _publish(self): out = PoseWithCovarianceStamped() out.header.frame_id = 'map' out.header.stamp = self.get_clock().now().to_msg() out.pose.pose.position.x = self.x[0] out.pose.pose.position.y = self.x[1] # Fill 2x2 position covariance block into 6x6 covariance array out.pose.covariance[0] = self.P[0,0] # Cov_xx out.pose.covariance[1] = self.P[0,1] # Cov_xy out.pose.covariance[6] = self.P[1,0] out.pose.covariance[7] = self.P[1,1] # Cov_yy self.pub.publish(out) def main(): rclpy.init() rclpy.spin(PersonTracker()) rclpy.shutdown()
The critical line is the gate check inside meas_cb. Any detection more than 1.5 m from the predicted position is silently rejected. This is the same IoU gate that SORT uses — just expressed in Euclidean distance rather than bounding-box overlap because we are working in 3D space. The gate simultaneously protects against ID-switches (a bystander 2 m away will be rejected) and causes track fragmentation (if the target accelerates quickly, they may briefly leave the gate and cause a miss).
WEAKNESSWhere tracking goes wrong in the field
The intuition is: set max_age higher and tracks survive longer occlusions without fragmenting. The failure mode: zombie tracks proliferate. In a scene with 5 people where 3 are momentarily behind an obstacle, a high max_age keeps 5 predicted tracks alive — all propagating forward with the motion model. When the 3 people reappear, there are now 8 things (5 old tracks + 3 new detections initially unmatched) competing for assignment, and the cost matrix is maximally confused. Production trackers use separate max_age values for high-confidence tracks (longer) vs. newly-born tracks (shorter) to manage this. SORT's single max_age parameter is a known simplification.
A common mistake: set the YOLOv8 confidence threshold high (say, 0.75) to reduce false positives. At low ambient light, during a turn, or when the person is partially behind furniture, the detector drops below 0.75 and produces no detection — even though the person is clearly visible and physically present. The tracker sees "no detection" and increments the miss counter. After max_age frames, the track is killed. The person then reappears and gets a fresh ID — track fragmentation. The correct approach is to run the detector at a lower confidence threshold (0.3–0.5) and use the tracker's gate to filter false positives, rather than relying on the detector's confidence alone. The detector and tracker have complementary noise structures; the tracker is the right place to gate spatial plausibility, not the detector.
When a field robotics team reports "the tracker lost the person," the tracker is nearly always working exactly as designed. The actual causes, in rough order of frequency: (1) the detector produced no detection at a critical frame due to threshold, lighting, or pose; (2) the sensor fusion from the previous stage produced a 3D position estimate outside the gate; (3) the target moved faster than the gate radius between frames (physically impossible at the sensor rate, but common when the sensor droppped frames). Diagnosing this requires logging the full predict-assign-update sequence including the raw detections, the predicted positions, the gate distances, and the cost matrix — not just the final track output. If your tracker node does not log these, you cannot diagnose failures. Add structured debug logging before field deployment.
KEY INSIGHTHow it connects
- 5.5 · Person Detection and 3D Localization: LIDAR + Camera Fusion — the fusion pipeline that produces the 3D person position consumed by the tracker. The tracker is the stateful layer that sits above the stateless fusion output. Track quality is bounded by fusion quality: a 0.3 m fusion error means the tracker's Kalman filter is always operating on measurements that noisy, regardless of how well the filter is tuned.
- 4.5 · The Extended Kalman Filter: Nonlinear Systems and Jacobian Linearization — the Kalman filter theory underlying SORT's bounding-box filter. SORT uses the linear KF (not the EKF) because bounding-box kinematics are linear. The predict-assign-update loop structure is identical to what you built in Lesson 4.5 for robot localization; only the state vector and observation model differ.
- 5.7 · Occupancy Grid Mapping: The Map Representation That Works → — the map built by SLAM Toolbox is the environment context that the behavior tree uses to plan the approach trajectory to the tracked target. The tracker publishes a target pose; the planner needs the occupancy grid to find a collision-free path to that pose. These two components — tracker and map — are the two missing links that connect Lesson 5.5's sensor output to navigation.
The tracking layer is the bridge between raw perception (detections at each frame) and the temporal state representation that planners need. Without it, you have a detector that tells you "there is a person here right now" — useful but insufficient for following. With it, you have a filter that tells you "person A has been at these positions, is currently moving in this direction at this velocity, with this uncertainty" — the minimal state representation that a downstream planner can consume.
GLOSSARYKey terms introduced
- SORT (Simple Online and Realtime Tracking)
- A minimal multi-object tracker combining a per-track linear Kalman filter (state: bounding-box centroid, scale, aspect ratio + velocities) with the Hungarian algorithm for frame-to-frame assignment, gated by IoU. Uses no appearance features. Fast but susceptible to ID-switches at occlusion boundaries.
- Hungarian algorithm
- An O(n³) combinatorial algorithm that solves the linear assignment problem: given a cost matrix C[i][j] (cost of assigning track i to detection j), find the minimum-cost one-to-one matching. The assignment step of SORT. Also called the Kuhn-Munkres algorithm.
- ID-switch (IDSW)
- A tracking failure in which the tracker assigns a detection of object A to the track of object B (or vice versa), causing the two tracks to swap identities permanently. The dominant failure mode of IoU-only trackers under occlusion.
- IoU gate
- A threshold applied to the cost matrix before running Hungarian assignment: track-detection pairs with IoU below the threshold are treated as inadmissible (cost = infinity). Prevents assignments across large spatial gaps; too tight causes track fragmentation, too loose causes false cross-target assignments.
- Track birth / track death
- Track birth: a new detection that is unmatched by any existing track spawns a candidate track; it is confirmed as a live track after receiving detections in min_hits consecutive frames. Track death: a live track that accumulates more than max_age consecutive missed-detection frames is deleted.
- DeepSORT
- An extension of SORT that adds a deep-learned 128-D appearance descriptor extracted from each detection crop. Assignment cost combines IoU and cosine distance in embedding space. Significantly reduces ID-switches on short occlusions at the cost of an appearance model inference pass per frame.
- Track fragmentation
- A tracking failure in which a single physical object is represented as two or more separate tracks with distinct IDs over its lifetime, because the track was killed during an occlusion gap and the object re-appeared with a new ID. Complementary failure mode to ID-switch.
- Kalman gate (distance gate)
- A threshold on the Mahalanobis distance (or Euclidean distance in simpler implementations) between a new detection and a track's predicted position, used to reject implausible assignments before running the Hungarian algorithm. Equivalent in function to the IoU gate but expressed in state-space distance rather than bounding-box overlap.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1SORT assigns detections to tracks using IoU. Why does this reliably fail at occlusion exit, rather than during the occlusion itself?›
During the occlusion, one track simply receives no detection and propagates its prior forward using the motion model. Nothing is assigned incorrectly — the track just ages. The failure occurs at occlusion exit because that is the moment a new detection appears that could plausibly match either the surviving track or the re-emerging track. When person A re-emerges from behind an obstacle, A's new bounding box position may be closer (higher IoU) to track B's predicted position than to track A's own predicted position — because track A's prediction has been propagating uncorrected during the occlusion and has drifted. The Hungarian assignment then gives detection-A to track-B. The ID-switch is geometrically determined by which predicted box happens to be spatially closer at the moment of re-emergence.
2SORT's Kalman filter uses an 8D state vector. What physical quantities are in the state, and why is a linear KF (not an EKF) appropriate for this application?›
The 8D state is [u, v, s, r, u̇, v̇, ṡ, ṙ]: center x-coordinate (u), center y-coordinate (v), scale/area (s), aspect ratio (r), and their first time derivatives. The motion model is x_{t+1} = F·x_t + noise where F is the identity-with-velocity block matrix — a purely linear relationship. The observation model is also linear: z = H·x + noise where H simply extracts the first four components. There is no nonlinearity anywhere in the model (compare with robot localization where the heading appears inside a trigonometric function in the motion model, requiring the EKF's Jacobian linearization). Because both the motion model and observation model are linear, the standard KF gives the exact optimal Bayesian solution — no approximation needed.
3FollowBot uses a single-target tracker, not a multi-object tracker. In what specific crowd scenario does the ID-switch problem still occur, and what would you need to add to prevent it?›
The failure scenario: the target person (A) passes behind an obstacle while a bystander (B) is nearby. Track A's predicted position drifts toward B's actual position during the occlusion. When A re-emerges, the distance gate accepts B's position as a valid update for track A (because B is closer to A's drifted prediction than A's true position is). From this point the tracker is following B. Prevention requires an additional disambiguation signal beyond geometry: an appearance descriptor (e.g., person's clothing color/texture encoded as a CNN embedding) so that detections of B can be rejected as "wrong appearance for track A," even when geometrically plausible. Alternatively, an active beacon (UWB tag, QR code on person's back) provides an identity signal that is immune to appearance ambiguity. Both require additional hardware or compute. The minimal hardware-free mitigation is to shorten the gate radius — but this causes track fragmentation in dynamic environments.
4A teammate proposes setting max_age = 50 (5 seconds at 10 Hz) so that FollowBot's tracker never loses the person during long occlusions. What is the failure mode this creates?›
With max_age = 50, the track remains alive for 5 seconds of no detection — propagating its position prediction using the motion model alone. Two failure modes emerge. First, prediction drift: the constant-velocity motion model will have diverged significantly from the person's true position after 5 seconds of uncorrected propagation. When the person reappears, their actual position may be outside the gate relative to the track's drifted prediction — causing the tracker to reject the correct detection and accept a nearby bystander's detection instead. Second, zombie track contamination: if another person enters the scene during the occlusion, there are now two entities in the assignment pool (the zombie track and the new detection), creating assignment ambiguity. The standard approach is to use a shorter max_age (1–2 seconds) combined with a recovery behavior (rotate-and-search) that actively re-acquires the target, rather than relying on the filter to survive long gaps.