The Sim-to-Real Gap: Taxonomy, Root Causes, and Honest Mitigations
The sim-to-real gap has four distinct failure classes — conflating them leads to applying the wrong mitigation and being surprised when the robot still fails on hardware.
🎯 By the end you'll understand
The four distinct categories of the sim-to-real gap — dynamics, sensor model, perception, and timing — and why each has a different root cause requiring a different mitigation. Why domain randomization closes some gaps but cannot close others. The honest effectiveness ceiling of every major mitigation technique. How to construct a per-gap checklist for FollowBot before a hardware bring-up, and how to interpret field failures by gap category rather than by symptom alone.
KEY INSIGHTThe gap is not one thing — and that matters
The number one mistake in sim-to-real transfer is treating the gap as a single continuous dial to be turned down. It is four separate gaps. Turning down one may have zero effect on the others.
Every team that has run a simulated robot and then tried to deploy on real hardware has felt the same thing: the policy or controller that worked perfectly in simulation fails on the real machine. The intuitive response is to make the simulation "more realistic" — add more physics fidelity, higher-resolution renders, better sensor models. Sometimes that helps. Sometimes it does nothing, because the failure is happening in a completely different part of the gap than the one you just improved.
Before you touch the simulator or the mitigation strategy, ask: which category of gap is causing this failure? A controller that drifts on hardware because friction was undermodeled will not be fixed by adding texture randomization to the renderer. A neural detector that fails because the sim camera omits lens distortion will not be fixed by adding joint compliance to the physics engine. The taxonomy comes first. Everything else follows from it.
CORE CONCEPTFour gap categories: a precise taxonomy
Robotics researchers and practitioners use the phrase "sim-to-real gap" loosely. Under that umbrella live four mechanistically distinct failure classes, each with different root causes, different diagnostics, and different (partially effective) mitigations.
The layers are important. Dynamics affects the physics engine. Sensor model affects how the simulator converts scene geometry into sensor readings. Perception affects what the neural representations have seen during training. Timing affects the control loop's stability margin. A policy trained in simulation with a perfect physics engine but a simplified sensor model will fail at layer 2 even though layer 1 was correct — and no amount of physics improvement will rescue it.
HOW IT WORKSGap category 1: the dynamics gap
The dynamics gap is the oldest and most studied of the four. It arises because the differential equations governing the simulator's physics engine are a simplified model of the real robot's mechanics.
Friction is the most common culprit. Simulators typically model Coulomb friction (a constant opposing force proportional to normal load). Real wheels on real surfaces exhibit static friction breakaway, Stribeck curves at low velocity, and velocity-dependent viscous drag that is load- and temperature-sensitive. The simulator's friction coefficient is a scalar; the real coefficient is a function of velocity, temperature, wheel compound, and surface cleanliness. Even a 20% error in friction propagates directly into velocity tracking error and heading drift.
Inertia errors are subtler. The simulator's moment of inertia comes from a CAD model — which assumes uniform material density, ignores wiring harnesses, battery position, and the actual mass distribution of the compute stack. A 10% inertia error causes the robot to turn faster or slower than commanded, which appears to the EKF as sensor noise but is actually a model mismatch.
Joint compliance and actuator lag matter especially for arms. A simulated joint is typically rigid; a real joint has elastic deformation from gearbox compliance and structural flexibility. The simulated actuator responds instantly to commanded torque; the real motor has electrical time constants (~2–10 ms for a brushless DC motor) and a current loop that limits torque rise rate. A policy trained to perform fast arm movements will apply forces that assume instantaneous response — on hardware, the arm resonates or overshoots.
System identification (measuring real robot parameters and updating the simulator's URDF inertial properties) can reduce the dynamics gap substantially for slow motions. For aggressive motions, the unmodeled effects dominate and no amount of careful measurement eliminates them. This is the honest ceiling of sys-id.
HOW IT WORKSGap category 2: the sensor model gap
Simulators generate sensor data by ray-casting against scene geometry. This produces geometrically correct readings, but geometrically correct is not the same as physically correct. Real sensors have failure modes that have nothing to do with geometry.
Wrong noise distribution. Most simulators add Gaussian noise to range measurements. Real LIDAR range noise is closer to Gaussian on valid returns, but the distribution of returns is not Gaussian — it includes a non-trivial fraction of ray drops (complete misses), multi-path reflections (phantom returns closer than the actual surface), and specular reflections (phantom returns at the mirror angle). A simulation that adds only Gaussian noise has the wrong noise model, not just the wrong noise magnitude. Changing the magnitude of Gaussian noise will never reproduce the multi-modal distribution of real LIDAR data.
Missing ray-drop. LIDAR rays drop on absorptive or retroreflective surfaces. Black clothing absorbs 785 nm near-infrared; polished metal returns a coherent specular reflection at the wrong angle; glass transmits. A sim LIDAR typically returns a range value for every ray. A real LIDAR for these surfaces returns inf. The EKF receives fewer measurements than the filter was designed for and, if the missing-return model is wrong, the filter diverges.
Absent lens distortion. The camera pinhole model used in most simulators is ideal — radial and tangential distortion are zero. Real camera lenses have both. A neural detector trained on ideal pinhole images sees real images that are slightly distorted, creating systematic localization errors at the image edges (where person legs often appear). This is a sensor model gap, not a perception gap: even a perfect real-image-trained detector will receive a distorted image that differs from what it was calibrated to expect unless the calibration is applied first. See Lesson 5.1 for the full distortion model.
This is a critical and commonly misapplied distinction. Domain randomization — randomizing noise magnitudes, lighting, textures — addresses uncertainty in the value of a parameter, not the absence of a physical phenomenon. If the simulator never generates ray-drop events, then randomizing the Gaussian noise magnitude does not expose the policy to missing returns. The policy remains brittle on any surface that causes ray drops. The correct fix for a structural sensor model error is to add the missing physical phenomenon to the simulator, not to randomize what is already there.
HOW IT WORKSGap category 3: the perception gap
The perception gap is the gap that neural networks care about most. It arises when the distribution of real sensor data differs from the distribution of simulated sensor data in ways that affect a learned model's output. A neural detector trained entirely on rendered images will fail on real images not because the physics is wrong or the sensor model is wrong — the failure lives in the representation space.
Texture and material mismatch. Real wood floors have grain direction, scuff marks, and reflectivity variation that real-time rendering engines approximate poorly. Sim humans wear flat-shaded garments from a small asset library; real people wear arbitrary clothing with fine texture detail. A person detector trained on rendered humans will see features — HOG gradients, CNN filter activations, YOLO anchor proposals — that differ systematically from real human features. The detector may have a high mAP on rendered test sets and a dramatically lower AP on real video.
Lighting and shadow shift. Simulators model direct illumination from point or area lights. Real indoor environments have complex indirect illumination: light bouncing off colored walls, fluorescent flicker at 100 or 120 Hz (which creates aliasing artifacts with typical camera shutter speeds), and specular highlights that move with camera angle. A detector trained without these cues is not robust to them — and adding them requires physically-based rendering, which is computationally expensive.
Domain shift for neural detectors. Even detectors pre-trained on large real-world datasets (COCO, Open Images) experience domain shift when deployed in specific environments: the indoor person appearance distribution differs from the COCO distribution, and fine-tuning on rendered data can hurt if the rendered distribution is further from the real deployment distribution than COCO was. This is contested in the literature and depends heavily on the renderer quality and fine-tuning recipe.
The perception gap is the gap that domain randomization was originally designed to address (Tobin et al., 2017, OpenAI). The core idea: if you randomize the visual appearance parameters of the simulation widely enough, the real world appears to the trained policy as "just another randomization" and the policy generalizes. This has been demonstrated to work for specific manipulator tasks. Whether it scales to complex visual environments remains an active research question — not a solved problem.
Domain randomization trains the model to be robust to many simulator configurations, hoping the real world falls within the randomized distribution. Domain adaptation uses a small amount of real data (paired or unpaired) to shift the model's learned representation toward the real domain — techniques like CycleGAN image-to-image translation or feature-level adversarial alignment. Neither is universally better: domain randomization requires no real data but risks high variance policies; domain adaptation requires real data collection, which is expensive and potentially dangerous for hardware.
HOW IT WORKSGap category 4: the timing gap
The timing gap is the most frequently overlooked of the four — and in some deployment scenarios, the most decisive. It arises because the simulation's compute schedule does not match the real robot's compute schedule.
Simulation runs faster than real-time. A physics simulator running on a fast server often executes faster than wall-clock time (10× real-time or more). The trained policy interacts with the simulation at a control frequency determined by the simulation's time step, not by real hardware latency. When deployed on hardware, the same policy runs on an onboard computer that may deliver only 10–15 Hz for a heavy neural inference path. The policy was trained assuming 50 Hz control; it receives 12 Hz. The dynamics are in a different regime and the policy is no longer stable.
Compute-bound latency. A YOLO-based person detector running on an NVIDIA Jetson AGX Orin takes 18–35 ms per inference frame. In simulation, the detector (or its surrogate) may run at 1 ms. The policy therefore acts on sensor data that is 17–34 ms staler than it expected. At FollowBot's maximum speed of 1.2 m/s, 30 ms of unexpected latency means the control decision is based on where the person was 3.6 cm ago — tolerable for slow following, non-trivial for turn tracking.
OS scheduling jitter. Real Linux (non-RTOS) scheduling introduces timing jitter of 1–10 ms per control cycle. The simulator is deterministic. A controller tuned to work with deterministic timing may oscillate when the timing has real-world jitter, especially if the controller gain is near the stability boundary. Using SCHED_FIFO or a real-time kernel patch reduces but does not eliminate this effect.
Before deploying any policy on hardware, measure the actual latency distribution of every node in your pipeline under real hardware load. Compare it to the simulation assumption. If the simulation used a 20 ms control period and your real pipeline delivers 60 ms average with 30 ms standard deviation, the timing gap will dominate everything else. No dynamics calibration or domain randomization will help until timing is fixed.
SEE ITFollowBot sim-to-real gap quantifier
The widget below models FollowBot's person-tracking error as a function of which gap sources are active. Toggle each of the six gap sources on and off; observe the cumulative effect on tracking error, see which gap category each source belongs to, and read the honest assessment of each mitigation's effectiveness. The worst combination reveals which gaps dominate — and which cannot be fixed by the mitigation you might reach for first.
The widget reveals an asymmetry that surprises most practitioners: the timing gap and perception gap together can dominate the dynamics gap, even though the dynamics gap is the one everyone talks about and for which the most tooling exists. On FollowBot's hardware (Jetson class compute, neural person detector), the timing gap alone contributes 0.08 m of steady-state tracking error — comparable to a 20% friction coefficient error.
IN PRACTICEMitigations — honestly assessed
Three major mitigation families exist. None closes the gap fully. Each closes a different portion of it. Applying the wrong one wastes engineering time; the failure will still happen in the field.
| Mitigation | Gap categories addressed | Effective for | Not effective for | Honest ceiling |
|---|---|---|---|---|
| Domain randomization | Perception ✓, some Sensor model ◑ | Texture/lighting variation; parameter-value uncertainty | Structural sensor model errors (absent phenomena); timing; dynamics | Works well for tabletop manipulation. Generalization to complex environments is contested. |
| System identification | Dynamics ✓ | Friction, inertia, joint compliance at slow speeds | High-speed dynamics; thermal variation; wear over time | Reduces dynamics gap 60–80% for quasi-static motions. Degrades at high speed and for arm dynamics. |
| Sim-to-real calibration | Sensor model ✓, Timing ✓ | Measuring and compensating real latency; adding physical sensor phenomena to sim | Cannot compensate for phenomena the sim cannot represent at all | Timing gap nearly eliminable if measured carefully. Sensor model gap reducible but not eliminated. |
| Real-data fine-tuning | Perception ✓ | Closing neural detector domain shift with a small real dataset | Requires real data collection; can overfit; doesn't help dynamics/timing | Typically most effective per engineering hour for perception gap. Safety risk during data collection. |
| Adaptive control | Dynamics ◑ | Online estimation of friction/inertia mismatch during deployment | Perception, sensor model, timing | Adds control complexity; convergence guarantees are limited for highly nonlinear systems. |
Notice that no single mitigation addresses all four gap categories. The practical implication: a real sim-to-real checklist must include at least one mitigation per gap category. A team that applies only domain randomization and calls the sim-to-real problem "solved" has addressed at most two of four categories — and only partially.
After applying every mitigation technique, a residual sim-to-real gap remains. The engineering question is not "how do we close the gap completely?" but "what is the acceptable residual gap for our task?" A person-following robot tolerating ±0.15 m steady-state tracking error can accept a larger residual than a surgical robot with ±1 mm tolerance. The honest answer to "is sim-to-real good enough?" is always task-specific and requires field testing — not simulation metrics.
System identification in practice for FollowBot
Sys-id for a differential-drive robot involves measuring the real friction and inertia parameters and updating the URDF's <inertial> block and the controller's dynamic parameters. A standard procedure:
import rclpy, math from rclpy.node import Node from nav_msgs.msg import Odometry from geometry_msgs.msg import Twist # Ramp the robot to a target speed, measure steady-state velocity error. # The difference between commanded and achieved velocity → friction coefficient. # Run at 3 speeds; fit a line through (v_cmd, v_actual) to separate viscous + Coulomb. class FrictionSysId(Node): def __init__(self): super().__init__('friction_sysid') self.pub = self.create_publisher(Twist, '/cmd_vel', 10) self.create_subscription(Odometry, '/odom', self.odom_cb, 10) self.v_samples = [] self.v_cmd = 0.0 self.create_timer(0.1, self.step) self.phase = 0 self.t = 0 def odom_cb(self, msg): self.v_samples.append(msg.twist.twist.linear.x) if len(self.v_samples) > 20: # keep last 2 s at 10 Hz self.v_samples.pop(0) def step(self): targets = [0.3, 0.6, 1.0] # m/s — three test speeds self.t += 1 if self.phase < len(targets): self.v_cmd = targets[self.phase] tw = Twist(); tw.linear.x = self.v_cmd self.pub.publish(tw) if self.t % 30 == 0: # 3 s per speed → steady state v_ss = sum(self.v_samples) / len(self.v_samples) self.get_logger().info( f'cmd={self.v_cmd:.2f} m/s achieved={v_ss:.3f} m/s err={self.v_cmd-v_ss:.3f}' ) self.phase += 1 else: self.pub.publish(Twist()) # stop self.get_logger().info('Sys-id complete. Update URDF friction and rerun in sim.') rclpy.shutdown()
The measured velocity errors at each commanded speed are fitted to the Coulomb + viscous friction model: Ffriction = μc + μv · v. The intercept gives the Coulomb coefficient (static breakaway); the slope gives the viscous coefficient. These values update the simulator's joint friction parameters in the URDF. Expect a 40–70% reduction in straight-line drift error after a careful sys-id run on a fresh, clean surface.
FOLLOWBOTFollowBot sim-to-real checklist
FollowBot's full Nav2 stack is now configured: A* global planner on the SLAM map, DWA local planner, costmap with LIDAR obstacle layer, behavior tree orchestrating detect-navigate-follow-recover. Before any hardware bring-up, a per-gap checklist must be signed off. This checklist is not optional — it is the difference between a field test that teaches you something and a field test that damages hardware.
Dynamics gap — signed off when
Friction sys-id has been run on the actual floor surface (tile, carpet, or gravel — the real deployment surface, not the lab floor). URDF inertial properties match a weighed and measured robot (with full compute stack and battery installed). Velocity tracking test shows <5% steady-state error at 0.3, 0.6, and 1.0 m/s. Joint compliance for the 6-DOF arm has been characterized at the arm's expected payload weight. DWA dynamic constraints (max linear velocity, max angular velocity, acceleration limits) match hardware-measured limits, not spec sheet values.
Sensor model gap — signed off when
The LIDAR simulator has been configured with a ray-drop model: empirically measure the false-return rate on representative surfaces (black mat, glass, metal) and set the simulator's dropout probability to match. Camera distortion parameters from real camera calibration (Lesson 5.1 procedure) have been applied to the simulation camera or to the perception pipeline's pre-processing step. Noise magnitude for the RPLIDAR A2 has been fit to real scan variance measurements (expected: σ ≈ 15–25 mm at 3 m). Camera exposure and gain settings in sim match the real camera's auto-exposure output for the deployment lighting.
Perception gap — signed off when
The YOLO person detector has been evaluated on at least 50 real images of people in the deployment environment wearing representative clothing. Detection precision and recall are documented (not assumed from COCO benchmarks). If recall <85% on real images, fine-tuning on a real dataset is required before deployment. The depth camera's person segmentation output has been visually confirmed to produce valid bounding boxes at the 0.8–3.5 m tracking range. Any domain adaptation step (CycleGAN, FeatureDA) has been verified not to reduce precision on the real test set.
Timing gap — signed off when
The full pipeline latency has been measured on the onboard hardware under full load (SLAM, EKF, person tracker, DWA, YOLO simultaneously): median and 95th-percentile latency for the sensor-to-cmd_vel path. If median latency exceeds 80 ms, the DWA planner's prediction horizon must be extended to compensate. The control loop frequency on hardware matches the simulation's assumed rate within 10%. A watchdog timer is in place: if cmd_vel is not received within 200 ms, the robot stops. LIDAR and camera topic timestamps match wall-clock time to within 5 ms (clock sync verified with ros2 run tf2_ros tf2_monitor).
The Stage 6 capstone requires a documented system architecture with an honest per-subsystem assessment: what each component does well in simulation, what the known sim-to-real gap is for that component, what mitigation was applied, and what residual error remains on hardware. This is not a checklist of completed tasks — it is a statement of known unknowns. A deployment that is honest about its residual gaps is safer than one that claims the gap is closed.
The FOLLOWBOT milestone for this lesson is the completion of the signed-off sim-to-real checklist in all four gap categories. The checklist becomes the entry criterion for Lesson 6.8's hardware bring-up procedure. Do not proceed to hardware bring-up with unsigned gaps — an unsigned perception gap means the person detector may fail silently in the field, creating a robot that moves at 1.2 m/s toward a person it can no longer see.
WEAKNESSWhere sim-to-real bites hardest
Domain randomization randomizes parameter values — noise magnitude, lighting intensity, texture color, friction coefficient. It cannot compensate for a physical phenomenon that is not modeled at all. If the simulator does not generate ray-drop events, randomizing noise magnitude cannot teach the policy to handle missing LIDAR returns. The policy learns that ranges vary — not that ranges can simply not exist. On hardware, the first surface with significant ray absorption will cause tracker divergence that was never seen in simulation, no matter how widely the noise magnitude was randomized. The correct diagnosis: this is a structural sensor model gap. The correct fix: add the missing physical phenomenon to the simulator.
Friction, LIDAR ray-drop rates, and camera lighting conditions all change significantly between a clean lab floor and a deployment environment. A sys-id performed on the lab's smooth epoxy floor will underestimate friction on carpet and overestimate it on polished concrete. A perception calibration done under controlled lab lighting will be wrong under the fluorescent flicker and colored wall reflections of the real deployment site. Sim-to-real calibration must be performed in the deployment environment, or must include explicit uncertainty bounds that are conservative enough for the deployment environment's expected range of conditions. "Works in the lab" is not a sim-to-real result — it is a lab result.
Timing gap failures are invisible in software-in-the-loop testing because the test harness typically runs at the simulation's time step, not at the hardware's real latency. They are also invisible in unit testing of individual nodes. The timing gap only manifests when the full pipeline runs on hardware under real load. By the time you discover that the YOLO inference takes 35 ms instead of the assumed 5 ms, the controller has been tuned for the wrong latency regime and must be re-tuned entirely. Measuring pipeline latency on representative hardware should happen before any other calibration, because every other calibration assumes a fixed timing regime. Get the timing right first.
KEY INSIGHTHow this connects
- 3.3 · Actuator Models and the Sim-to-Real Dynamics Gap — the deep treatment of the dynamics gap: how friction, inertia, and actuator lag are modeled in simulation, how they are measured on hardware, and what the residual error looks like after sys-id. The present lesson extends that to a four-category taxonomy; go back to 3.3 for the dynamics gap's math.
- 5.1 · Camera Geometry: The Pinhole Model, Distortion, and Why Calibration Is Not Optional — the sensor model gap for the camera is rooted in lens distortion: the sim camera is an ideal pinhole and the real camera is not. Lesson 5.1 derives the distortion model and the calibration procedure that feeds into the sensor model gap mitigation above.
- 6.5 · The Nav2 Stack: Architecture, Lifecycle Nodes, and Failure Modes — the Nav2 stack is the deployment context within which all four gap categories manifest. Timing gap failures appear as missed controller deadlines; dynamics gap failures appear as costmap violations; sensor model gap failures appear as phantom obstacles. The failure mode analysis from 6.5 and the gap taxonomy from this lesson together produce the FollowBot checklist above.
- 6.8 · Hardware Bring-Up, Calibration, and Field Diagnostics → — the next lesson executes the bring-up procedure assuming this lesson's checklist is signed off. Every item on that procedure traces back to a specific gap category and a specific mitigation from this lesson.
The sim-to-real gap is not a problem you solve once and set aside. It is a continuous accountability framework: for every new subsystem added to the robot, for every new deployment environment, for every hardware upgrade, a new gap assessment is required. The taxonomy introduced here is the scaffolding that makes that assessment tractable rather than ad hoc.
GLOSSARYKey terms introduced
- Sim-to-real gap
- The family of discrepancies between a robot's behavior in simulation and its behavior on real hardware. Correctly understood as four mechanistically distinct gap categories (dynamics, sensor model, perception, timing), not a single monolithic quantity.
- Dynamics gap
- Mismatch between the physics engine's model of friction, inertia, compliance, and actuator response and the real robot's mechanical behavior. Addressed by system identification. Reducible but not eliminable.
- Sensor model gap
- Mismatch between the simulator's sensor model and real sensor physics — specifically, physical phenomena absent from the simulator (ray-drop, lens distortion, multi-path reflections). Domain randomization of existing parameters does not address absent phenomena.
- Perception gap
- Mismatch between the distribution of simulated sensor data and real sensor data in the neural representation space. The target gap for domain randomization and domain adaptation techniques.
- Timing gap
- Mismatch between the simulation's compute timing (often faster than real-time, deterministic) and real hardware timing (bounded by compute capacity, subject to OS scheduling jitter). Causes the deployed policy to operate at a different effective control frequency than it was trained for.
- Domain randomization
- A training-time technique that randomizes simulation parameters (textures, lighting, noise magnitudes, friction) widely enough that the real world is treated as another sample from the randomized distribution. Addresses perception gap and parameter-value uncertainty in sensor models. Does not address structurally absent phenomena or timing gaps.
- System identification (sys-id)
- The empirical measurement of real robot mechanical parameters (friction coefficients, inertia tensors, joint compliance) for use in updating the simulator's physics model. Primary mitigation for the dynamics gap.
- Ray-drop
- A LIDAR failure mode in which a laser pulse is absorbed by or specularly deflected away from a surface, returning no range measurement. Real LIDARs experience ray-drop on absorptive (dark clothing), retroreflective, and glass surfaces. Most simulators omit this phenomenon by default.
ACTIVE RECALLCheck your understanding
Answer in your own words first, then reveal.
1FollowBot's person detector works well in simulation but has poor recall on real hardware. You apply domain randomization to the simulator's textures and lighting. The recall on real hardware improves slightly but remains unacceptable. What is the most likely remaining cause and what should you try next?›
The most likely remaining cause is a structural sensor model gap or a perception gap that domain randomization could not close. Two candidates: (1) if the camera has significant lens distortion that the sim camera omits, the detector sees systematically different edge patterns at image boundaries on real images versus sim images — domain randomization of textures does not help because the geometric distortion is absent from the sim, not just noisy; (2) the sim-rendered human appearance distribution (flat-shaded garments, limited asset variety) may still be far enough from real human appearance that wider randomization does not bridge the gap. Next steps: apply camera distortion calibration to the real pipeline (or add distortion to the sim camera); and collect a small real-image fine-tuning dataset. Real-data fine-tuning is typically more effective per engineering hour than extending domain randomization when the gap is in appearance distribution rather than parameter-value uncertainty.
2Why does a 20% friction coefficient error affect FollowBot's heading accuracy more than its straight-line speed accuracy?›
Straight-line speed error from friction is symmetric — both wheels experience the same error, and the EKF's velocity measurement corrects for steady-state speed error over a few control cycles. Heading error from friction is asymmetric and accumulating: even a small difference in effective friction between the left and right wheel (due to floor surface variation, tire wear, or load distribution) produces a persistent yaw rate that the EKF integrates into a growing heading error. At 1.0 m/s with a 20% friction error that creates a 2% left-right asymmetry, the robot accumulates approximately 0.7° of heading error per meter of travel — enough to cause a 0.6 m lateral offset after 50 m, which is a common indoor traversal distance. Heading error is harder to correct because it requires the EKF to observe a loop closure or a fixed landmark to re-zero — the DWA controller cannot self-correct heading drift without external position reference.
3The sim-to-real gap taxonomy places "timing" as a separate category from "dynamics." Why is it not sufficient to just model the timing gap as an additional time delay in the dynamics model?›
Treating the timing gap as a fixed delay in the dynamics model would be sufficient only if the timing were deterministic and constant. Real hardware timing has two components that a fixed-delay dynamics model cannot represent: (1) variable latency — the DWA planner's computation time spikes from 15 ms to 80 ms when the costmap becomes complex, and YOLO inference time varies with frame content (fewer detections = faster); a fixed delay cannot represent this variance, and variance in the control period destabilizes controllers tuned for a fixed period; (2) OS scheduling jitter — Linux scheduler introduces 1–10 ms of random per-cycle jitter that is statistically independent of the robot's state, not a function of dynamics. A dynamics model with a fixed delay is also not general: every hardware configuration has a different timing distribution, requiring re-identification for each deployment target. The timing gap is therefore treated as a separate category with its own measurement and mitigation procedure.
4After signing off all four gap categories in FollowBot's checklist, you deploy and the robot still fails — it loses the person behind a glass partition. Which gap category was not adequately closed, and how would you update the checklist?›
This is a sensor model gap failure — specifically, missing ray-drop modeling for transparent surfaces. Glass transmits 785 nm near-infrared (the RPLIDAR A2's wavelength), causing the LIDAR to see through the partition and into the space behind it rather than detecting the partition itself. The checklist item for the sensor model gap said "empirically measure false-return rate on representative surfaces." If glass was not included in the test surfaces, the ray-drop model was incomplete for the deployment environment. Update the checklist: add glass partitions (and other common office surfaces — polished metal, whiteboard, safety mirrors) to the ray-drop characterization protocol. Measure the transmission fraction at 785 nm for each surface type. Update the simulator's ray-drop model to include transmission events (where the ray passes through and returns a range to whatever is behind the surface) in addition to absorption events (no return). Revalidate the SLAM and person tracker with the updated sensor model before the next field test.