Validation report (v1.0, brought up to date 2026-11-03)¶
This report is the roadmap §9 deliverable “validation report per domain”. It collects, for each
domain, what the model is checked against, the tolerance, the result and the test that checks it.
Sources: the “Validation” and “Status” sections of docs/design/domains/*.md,
docs/design/huge-worlds.md, the acceptance tables in docs/release/v0.2.0.md to v0.7.0.md,
and docs/plan/progress.md.
What “validated” means here¶
A model counts as validated against a reference when a test compares it with one of:
a closed form (an analytic solution, or a monolithic high-accuracy solution);
a published reference (a report, standard or dataset, cited below as the docs cite it);
measured data from a real robot or bench (sim-vs-real).
Each check states its tolerance and names the test. “Met” means the test passes at that tolerance on the development machine (GitHub Actions are disabled; the tests run locally).
Sim-vs-real against measured robot data mostly does not exist yet. Only the battery is
compared with measured data (a published cell dataset). Every other recording that the domain
docs plan (AMR odometry, a BLDC bench, a motor heat run, a magnetometer near a known source,
Wi-Fi RSSI in a building, a drone flight log) waits for open question Q16: which hardware is
available, and whether recordings can be published under an open licence
(docs/plan/open-questions.md). Most domains are therefore checked against closed forms and
published curves only, which is risk R12 in docs/plan/risks.md.
Summary¶
Domain |
Checked against |
Tolerance / result |
Test |
Sim-vs-real (measured data) |
|---|---|---|---|---|
Rigid body |
Closed-form mechanics; backend agreement MuJoCo vs Jolt; contact force and joint reaction on both (Newton) |
Energy drift < 0.1 % over 10 s; Coulomb angle within 1°; inertia 1e-9; backends within 2 %; weight held within 0.5 %, a swinging pivot’s load within 1 % |
|
none yet (AMR square drive, Q16) |
Electrical |
Closed forms; LG HG2 cell data (Kollmeyer et al.); ngspice |
0.1–0.5 % closed forms met; battery 5.4–6.2 mV RMS (46–89 % SoC), 21.5 mV at 24 % (target < 20 mV, not met); drive cycles at 25 °C (fitted to UDDS, unseen LA92 11.7 mV, US06 20.1 mV, just over), 39–145 mV at 40 to 0 °C (no temperature dependence) |
|
battery: published cell data; BLDC bench: none yet (Q16) |
Thermal |
Closed forms |
0.1 %, 0.5 %, 0.5 K met; energy to 1e-9 |
|
none yet (motor heat run, Q16) |
Rotor aero |
NACA Report 640; momentum theory; classical H-force; Cheeseman–Bennett; NeuroBEM’s vortex-ring fit; Hsiao and Chirarattananon’s ceiling balance; downwash: Bauersfeld et al. 2024, Tanner et al. 2015, Kiran et al. 2025, own OpenFOAM |
CT within 8 %, CP within 6 %; H-force 1 %; ground and ceiling effect exact to their curves; stacked drones 53 % kept at 4 l against 65 % measured (12 points short) |
|
none yet (flight log, Q16) |
Live air (lattice Boltzmann) |
Momentum theory; own OpenFOAM hover cases |
Momentum through a plane within 2 %; at 3 cm the centreline within 20 % and the outwash within 6 % of OpenFOAM; at 4 cm the outwash 19 % low |
|
none (OpenFOAM is a model too) |
Aerodynamics (drag, wind) |
Closed form; Dryden (MIL-F-8785C) |
Terminal velocity 0.5 %; turbulence variances 5 %; periodogram check owed |
|
none yet |
Weather |
Koschmieder, Beer–Lambert, Marshall–Palmer, ICAO density |
Exact; rain rate 5–18 % (target 5 %, not met; test at 20 %) |
|
none yet |
Sensor weather effects |
Lidar equation, Koschmieder veil |
Trends tested; fog clutter uncalibrated |
|
none yet (fog/rain chamber data, DENSE) |
EMI |
Closed forms (conductor, dipole, far field); aliasing |
1e-6 relative; 60 → 40 Hz alias exact; 8.08 µT at 9.9 m from 400 A |
|
none yet (Q16) |
Comms |
Friis, |
Friis, noise floor, BPSK 1e-5 point, CAN transitions exact |
|
none yet (RSSI recording, Q16) |
Sensors: lidar |
Analytic scenes; Tier 0 vs Tier 1 |
Tiers within 1 mm: worst 0.16 mm (Iris Xe), 0.31 mm (llvmpipe), same shape 100 % |
|
none yet |
Sensors: camera |
Pinhole model via OpenCV; EMVA 1288 linear model |
0.066 px mean, calibration RMS 0.07 px (< 0.1 px); photon-transfer variance within 3 % |
|
none yet |
Sensors: radar |
Radar equation; RCS of a plate |
RCS within 3 %; Doppler; detection range |
|
none yet |
Sensors: IMU |
Allan deviation of a 4-hour static record |
Within the estimator’s confidence band (result not stated in the docs) |
|
none yet |
Sensors: GNSS |
|
Exact against |
|
none yet |
Sensors: magnetometer |
World Magnetic Model 2025, NOAA test points |
Within 0.001 nT (worst 7e-4 nT) |
|
none yet |
Sensors: barometer |
ICAO standard atmosphere |
ICAO table values |
|
none yet |
Vehicle dynamics |
v²/(2μg); linear bicycle model; Magic Formula shape |
Braking 3.2 % (5 %); yaw rate 1.4 % (3 %); MF peak, slope, asymptote exact |
|
none yet |
Traffic and scenarios |
IDM equilibrium; light cycle; sample scenario timeline |
IDM, lights, determinism met; fundamental diagram owed |
|
not applicable / none yet |
Huge worlds |
Same world at the origin; GeographicLib |
Bit-identical poses at 500 km; rebase within 1e-9 m (MuJoCo), 1e-4 m (Jolt) |
|
not applicable |
Radiation |
— |
Not built; |
— |
none |
Hydrodynamics |
Buoyancy, drag, a sphere’s added mass (Lamb), currents and linear (Airy) waves ( |
released sphere’s rise within 3 % of g(ρV − m)/(m + ½ρV); drift with a current within 0.02 m/s; heave on long waves within 1 cm of the amplitude. Not Fossen’s full model (no rotational added inertia, no Coriolis terms, no radiation) |
|
none measured |
Since v1.0 (to 2026-11-03)¶
tools/ci/realism.sh runs every check in this section that has measured data or a published
reference, and writes the numbers to one page (realism.md); the nightly workflow’s realism
job runs it.
The models added after the v1.0 release, each with its reference, result and test. Where the reference is our own OpenFOAM run, that is a check between two models, not against the air.
Model |
Reference |
Result |
Test |
|---|---|---|---|
Rotor downwash, far jet |
Bauersfeld et al. 2024 (arXiv:2403.13321), measured multirotor jets |
fitted to it; its momentum flux carries the thrust within 5 %; our OpenFOAM hover centreline about 10 % under it |
|
Rotor downwash, ground outwash |
Tanner et al. 2015 (NASA), 1.9 V_h at 1.7 R |
fitted to it at one rotor height (1 R); between 1 R and 2 R the model’s peak rises where the measurement fades (rotor_aero.md §6) |
|
Drone under a drone |
Kiran, Ayanian and Breuer 2025 (RSS), stacked Crazyflies at constant voltage |
76 % of the thrust kept at 4 l (measured 65 %): 11 points over since wakes are averaged over each disc (53 %, 12 under, when sampled at the hub); 87 % at 19 l (measured 95–98 %): 8–11 under; constant voltage adds under one point |
|
A wall beside a drone |
own OpenFOAM, four gaps |
gain 0.0173 fitted; its fade length 3.5 ± 2.7 D is poorly fixed (Tanabe et al. say 1.5 D) |
|
Live air box ( |
momentum theory; own OpenFOAM hover at 2 m and 0.5 m |
momentum within 2 %; disc speed at its centre 23 % over ideal (blurred edge); at 4 cm through-disc air 15–20 % under OpenFOAM’s, the low-hover fountain 5.8–7.4 m/s against 7.1 m/s |
|
Vortex-ring state ( |
Bauersfeld et al. 2021 (NeuroBEM, RSS) eqs. 18–19, after Hoffmann et al. 2007 |
the fit exactly; thrust joins hover’s within 0.1 %; a step of 10 % of hover thrust into the windmill-brake state |
|
Ceiling effect |
Hsiao and Chirarattananon 2019 (IEEE/ASME TMech) momentum balance |
exact to it; their α0 = 1.6 is for a 23 mm propeller (TODO(verify) for larger ones) |
|
Contact force and joint reaction, Jolt |
Newton’s laws; MuJoCo |
box weight within 0.5 %; swinging pivot 11.348 N against 11.349 N (energy) and MuJoCo’s 11.349 N |
|
Wind tunnel ( |
shear-wave decay; Schiller–Naumann sphere; Hoerner plate and cube; Ahmed body (SAE 840300) |
shear wave 1 %; sphere 25 % over; plate 0.97 and cube 1.03 against 1.17 and 1.05; Ahmed body over four times too high (Cd 1.26 on 8M cells against 0.29) |
|
OpenFOAM cases ( |
Ahmed body at 25° |
Cd 0.266–0.288 against 0.2875–0.298, but 8 % apart between two meshes; 35° not trustworthy yet |
cfd.md §5 |
Components between buses (RFC 0010) |
formulas; ngspice |
Lagrange within 0.5 % of the formula, ngspice within 0.1 %, on random circuits |
|
Walking humans |
social force (Helbing and Molnár 1995) as traffic uses it |
behaviour tested (they walk, steer, stop); not against measured crowds |
|
Battery on drive cycles |
LG HG2 UDDS, US06, LA92 (Kollmeyer et al., CC BY 4.0) |
fitted to UDDS: LA92 11.7 mV, US06 20.1 mV RMS (target 20 mV; US06 not met by 0.1 mV); at 0 °C 145 mV |
|
Planar pushing (MuJoCo, Jolt) |
Yu et al. 2016 (IROS), 2000 measured pushes of rect1 on four surfaces; the quasi-static model they compare with (Lynch et al. 1992) |
MuJoCo within 6 mm and 4–7° of the quasi-static model; 3–41 mm and 10–17° from the measured means (as the model is). Jolt does not turn the block (2° against 75°): its linear and torsional friction have separate budgets |
|
Quadrotor rotors in flight |
NeuroBEM flights (Bauersfeld et al. 2021), 13 held-out segments, measured motor speeds and motion |
|
|
Fossen hydrodynamics |
BlueROV2 Heavy coefficients identified in a pool (von Benzon et al. 2022, CC BY 4.0) |
top speeds at full thrust 0.729/0.626/0.713 m/s against their 0.72/0.63/0.71 (their model, itself within ~10 % of the pool); start acceleration F/(m + m_a) within 3 % |
|
Thermal camera |
Incropera et al. Table 12.1 black-body fractions; Planck integrated directly; Stefan–Boltzmann; Swinbank 1963 clear sky |
fractions within 1e-5, band radiance within 1e-9; a 350 K node of ε 0.95 read at its apparent temperature within the 50 mK noise; scene temperatures are assumptions (no sun, no atmosphere) |
|
Models with no reference yet¶
Said plainly, so nobody reads more into these than is there:
Dust lifted by outwash: the threshold is Bagnold’s, but how much dust rises (the concentration curve) is our own choice, not checked against anything.
The live box’s unsteadiness: the rotor speeds move about 1 % in hover with it; no measured spectrum to compare.
Fountain lift on the frame in the fitted model (rotor_aero.md §2.8): fitted to our own OpenFOAM runs at five heights only (within 0.23 N), no measurement.
Hydrodynamics without a vehicle’s own coefficients: a body not in
--hydro-coefficientsstill takes a sphere’s added mass and cell drag; wave radiation is not modelled.Lidar intensity, GNSS error statistics, sensor timing histograms: no evidence found (Sensors table below).
Ultrasonic cone (the
lagrange_tofplugin): a geometric cone and dead zone, no datasheet comparison.Realistic camera look: a picture, not a radiometric model; only the pinhole geometry and noise are checked.
Thermal camera scenes: the radiometry is checked, but what a surface’s temperature is (ambient for anything without a thermal node, no sun, no sky reflection, no atmosphere) and a person’s skin temperature are our assumptions; no thermal image of a real scene to compare.
Radiation: not built.
Jolt’s contact force on a group of fixed bodies goes to the group’s root body; which body of the group was touched is not known.
Rigid body¶
Source: domains/rigid_body.md §7 (the doc has no status section; tests confirmed in
modules/rigid_body/mujoco_backend/test/test_mujoco_validation.cpp and
server/test/test_backends.cpp).
Check |
Reference |
Tolerance |
Test |
|---|---|---|---|
Free fall, pendulum, double pendulum energy drift |
closed form or high-order integration |
drift < 0.1 % over 10 s at 1 ms |
|
Sliding block on an incline |
Coulomb, closed form |
stick/slip angle within 1° |
|
|
analytic two-link arm inertia matrix |
1e-9 relative |
|
Determinism |
two runs, same seed |
bit-identical state hash (MuJoCo) |
|
Backend agreement, MuJoCo vs Jolt |
the same scene on both |
arm6 joint-space inertia within 2 %, its fall over 0.3 s within 2 % (+5 mrad), with armature within 5 % (+15 mrad); the sample AMR within 0.02 m/s and 0.02 rad/s |
|
Height fields (v0.7) |
the same grid read by both backends |
balls over a raised quarter rest within 1 cm of it |
|
Reported difference: on Jolt’s rigid contacts the AMR’s drive wheels slip while the drive ramps up, about 0.4 m behind after 6 s. PhysX is not built (the owner’s v0.6 decision), so the three-backend comparison in §7 covers MuJoCo and Jolt only.
Owed: the AMR drive test, sim-vs-real: recorded wheel odometry of a real differential-drive robot driving a square; “dataset to be recorded or sourced (open question Q16)”. Tolerance: position error after 10 m within the spread of repeated real runs.
Electrical¶
Source: domains/electrical.md §8–§9, v0.3.0.md, v0.4.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
RL step response, d and q |
closed form |
0.1 % |
met (exact up to rounding) |
|
DC motor no-load speed, stall current, torque-speed line |
closed form from R, Kt, V |
0.5 % |
met |
|
Coupled motor + inertia |
monolithic 1 µs solution |
2 % RMS |
met for inertias 6e-7 to 1e-2 kg·m², no growth; the quadcopter’s drive at 50 µs on 1e-5 and 9e-5 kg·m² rotors |
|
Battery ECM pulse |
closed form; LG HG2 cell, Kollmeyer et al., doi:10.17632/b5mj79w5w9.2, CC BY 4.0 (HPPC and C/20 at 25 °C) |
< 20 mV RMS per cell |
closed form exact; 5.4 to 6.2 mV RMS on held-out blocks from 46 to 89 % SoC; 21.5 mV at 24 % (not met) |
|
Energy balance over a drive cycle |
stored + work + losses |
0.5 % |
met |
|
Netlist round trip |
ngspice (averaged sources) |
bus voltage 1 % |
met: within 0.05 % (sag within 0.16 % RMS) |
|
Brownout scenario |
scripted stall on a low battery |
trips and recovers |
met |
|
Below about 35 % state of charge r0 and the OCV slope rise; a state-of-charge-dependent r0 is the next step. The battery is checked at 25 °C only. The Modelica export is checked for structure only (no OpenModelica on the machine).
Owed: sim-vs-real against a bench recording of a real BLDC drive (phase current, bus voltage, speed; “to be recorded; Q16”); the battery below 35 % state of charge; the drive-design review of the motor constants (RFC 0002 resolution 2).
Thermal¶
Source: domains/thermal.md §7–§8, v0.3.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
Single node cooling |
closed-form exponential |
0.1 % |
met |
|
Two-node network step |
closed-form two-pole response |
0.5 % |
met |
|
Radiative equilibrium of a sunlit plate |
closed form |
0.5 K |
met |
|
Energy conservation |
stored energy and flows |
0.1 % |
met, every step to 1e-9 |
|
Motor heat run |
closed-form steady state of a motor linked to the chassis |
— |
met |
|
Owed: the comparison with “a published motor thermal test with winding temperature rise at constant load, or a recorded bench test (Q16)”, within 3 K at steady state.
Rotor aero and aerodynamics¶
Source: domains/aerodynamics.md §7–§8, v0.4.0.md, v0.5.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
|
NACA Report 640 (1938, public domain), propeller 5868-9, two blades, Clark Y; section data from NACA Report 502 |
CT 8 %, CP 6 % (or 0.003) over the attached-flow range |
15°, J 0.3–0.7: CT 2–4 %, CP 2–5 % (7 % at J = 0.2); 25°, J 0.5–1.1: CT 0–7 %, CP 0–5 %; zero thrust at J = 0.82 and 1.29, as measured |
|
|
ideal rotor, figure of merit 1 |
3–4 % |
met |
|
|
|
1 % |
met |
|
|
moving through still air = standing in the wind; quadcopter in 8 m/s with gusts |
1e-12; within 0.1 m |
holds within 9 cm (6 cm in height) |
|
Static thrust and torque |
closed form; published propeller data |
5 % |
closed form exact; published data owed |
|
Ground effect |
Cheeseman–Bennett; published small-rotor measurements |
10 % for z/R 0.5–3 |
Cheeseman–Bennett exact for z/R 0.5–3; measurements owed |
|
Terminal velocity |
closed form |
0.5 % |
sphere within 0.5 % |
|
Turbulence spectra |
Dryden / von Kármán PSDs |
95 % band of the periodogram |
variances within 5 % of σ² over 4000 correlation times, longitudinal autocorrelation within 0.03 of e^(−τV/L); periodogram test owed |
|
Wind grids |
the grid’s own value |
exact |
met |
|
Geometry and NACA 640 measurements were read off the report’s charts (chord about ±1.5 %, blade angle ±0.3°, CT and CP about ±0.001); nothing was fitted to the propeller data. Below J ≈ 0.5 at 25° the inner blade stalls and the model has no stall-delay correction, so that range is not tested.
Owed: small-propeller test data (the UIUC database states no licence); measured ground effect; stall delay; the Dryden periodogram check; hover power and step response against “a flight log of a comparable real vehicle (to be sourced; Q16)”.
Weather¶
Source: domains/weather.md §7–§8, v0.5.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
Koschmieder and Beer–Lambert |
closed form |
exact to rounding |
met |
|
Moist air density |
ICAO sea-level density (dry air); Alduchov and Eskridge (1996) |
— |
within 5e-4 (as the source states it) |
|
Rain extinction |
Marshall–Palmer integral |
— |
met |
|
Rain rate from the drop distribution |
Marshall–Palmer with Atlas’s fall speeds |
5 % |
not met: 5 to 18 % more rain; the test checks 20 % |
|
Lidar detection range in weather |
lidar equation |
— |
tested |
|
Lidar clutter |
fog backscatter; Marshall–Palmer drops |
trends |
tested: rain clutter about 1 % of beams at 2 mm/h, 5 % at 25 mm/h |
|
Camera contrast in fog |
Koschmieder |
5 % |
veil tested |
|
Fields |
analytic fields, OpenFOAM-format round trip |
— |
trilinear exact; linear wind within 1e-5 |
|
The Kim model’s wavelength table is TODO(verify) against Kim, McArthur and Korevaar 2001. The fog clutter gain is uncalibrated.
Owed: lidar range in fog against “published lidar-in-fog chamber measurements (candidate: results from the European DENSE project)”, within 20 %; rain false returns against published rain-chamber statistics; the wet-friction literature range check; files from a real OpenFOAM case.
EMI¶
Source: domains/emi.md §7–§8, v0.5.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
Straight conductor and dipole fields |
closed form |
1e-6 relative |
met |
|
Far field of a transmitter |
closed form (Friis) |
1e-6 relative |
met |
|
Mains aliasing |
60 Hz sampled at 100 Hz gives 40 Hz |
peak location exact |
met |
|
Sample scene |
|
— |
8.08 µT RMS at 9.9 m from 400 A at 60 Hz |
|
The grid frequency is a site parameter with no fallback; the sample states 60 Hz in the scene.
Owed: field strengths near real lines against published measurements and guideline documents (“for example ICNIRP reference material”); the end-to-end recording from a real robot near a known source (Q16), target heading-error amplitude within 20 %.
Comms¶
Source: domains/comms.md §7–§8, v0.5.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
Free-space path loss |
Friis equation |
1e-9 relative |
met |
|
Thermal noise floor |
|
exact |
met |
|
Packet error rate curves |
AWGN bit-error expressions |
1 % |
BPSK 1e-5 point and Q(3) met |
|
CAN state machine |
ISO 11898-1 fault confinement |
exact |
met (transmit error counter only) |
|
Walls and shadowing |
wall counting; Gudmundson shadowing |
— |
wall count and spread met |
|
Owed: indoor parameters traced to ITU-R P.1238 (TODO(verify)); the end-to-end RSSI recording of an AMR driving away from an access point in a real building (Q16); the MAVLink bridge over links.
Sensors¶
Source: domains/sensors.md §2, §7, §8, v0.2.0.md, v0.3.0.md, v0.4.0.md, v0.6.0.md.
Sensor |
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|---|
Lidar |
geometry |
analytic scenes (plane, sphere, corner) |
< 1 mm before noise |
casts tested per shape |
|
Lidar |
height fields (v0.7) |
the field’s own surface ( |
1e-9 m |
met, vertical and slanted rays |
|
Lidar |
tier agreement |
Tier 0 vs Tier 1 |
same hit > 99.9 %; 1 mm |
worst 0.16 mm (Iris Xe), 0.31 mm (llvmpipe); same shape 100 % |
|
Lidar |
intensity |
Lambertian target |
2 % |
no evidence found |
— |
Camera |
projection |
checkerboard, OpenCV calibration |
< 0.1 px |
0.066 px mean; intrinsics RMS 0.07 px |
|
Camera |
noise |
photon-transfer curve (EMVA 1288 linear model) |
3 % |
met |
|
Camera |
lens, vignetting, rolling shutter, exposure |
OpenCV’s lens formulas; cos⁴θ; skew of a vertical line under a pan |
rolling shutter 1 px |
tested |
|
Radar |
RCS, Doppler, range |
σ = ρ·A; radar equation |
— |
plate RCS within 3 %; 10 m/s Doppler; range |
|
IMU |
noise |
Allan deviation of a 4-hour static record |
estimator’s confidence band |
test present; result not stated in the docs |
|
GNSS |
position |
|
exact |
met |
|
GNSS |
error statistics and correlation time |
— |
5 % |
no evidence found |
— |
Magnetometer |
main field |
World Magnetic Model 2025 ( |
0.001 nT |
worst 7e-4 nT |
|
Barometer |
pressure, temperature |
ICAO standard atmosphere (troposphere) |
table values |
met |
|
All |
timing |
rate, latency, jitter histograms |
1 base tick |
no evidence found |
— |
Radar and the camera model run on the CPU on every tier; hardware ray-query versions are owed.
Owed: sim-vs-real per release against “public datasets with ground truth and sensor specifications (candidates: an indoor 2D-lidar AMR dataset for v0.2; an automotive dataset for v0.6; choose and check licenses)”; a published RCS reference; radar rain attenuation and multipath; camera motion blur, a light model and demosaicing; GNSS outages and multipath; crustal field and hard- and soft-iron effects.
Vehicle dynamics¶
Source: domains/vehicle_dynamics.md §7–§8, v0.6.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
Braking distance from 20 m/s |
|
5 % |
3.2 % long |
|
Steady-state cornering |
linear bicycle model, understeer gradient |
3 % |
1.4 % |
|
Magic Formula |
its own peak, slope and asymptote |
exact |
met |
|
Ride height, static loads; gear shifting; ideal substitute |
— |
— |
met |
|
The sample tyre assets/tyres/generic_205_55R16.tir is illustrative.
Owed: a comparison with a published MF parameter set (“no openly licensed .tir was found”);
the ISO 4138, ISO 3888-2 and ISO 7401 manoeuvres as scenarios; the drivetrain energy balance
(1 %, no evidence found); sloped ground; the kinematic suspension; sim-vs-real against “a public
vehicle-dynamics dataset with IMU, wheel speeds and steering (to be selected; license check)”.
Traffic, OpenDRIVE and OpenSCENARIO¶
Source: domains/traffic.md §7–§8.1, v0.6.0.md.
Check |
Reference |
Result |
Test |
|---|---|---|---|
IDM equilibrium gap |
Treiber, Hennecke and Helbing 2000 |
met |
|
Light cycle; no entry on red over three minutes |
the configured cycle |
met |
|
Determinism ( |
same seed |
identical tracks |
|
Cut-in scenario |
its own triggers |
lane change at 3.5 s, halfway at 5 s, in lane at 6.6 s, 8 m/s at 7.5 s, ended at 12 s |
|
OpenDRIVE geometry |
closed forms per road type; lanes meeting end to start |
met |
|
Pedestrian walking speed 1.34 m/s (Weidmann 1993); social force after Helbing and Molnár 1995. SUMO and esmini are not built (the owner’s v0.6 decision), so the TraCI and SUMO alignment checks do not apply.
Owed: the pedestrian fundamental diagram against the literature; scenario conformance on the ASAM OpenSCENARIO example scenarios and a comparison with esmini; catalogs; OpenDRIVE superelevation, lane heights, objects and signals.
Huge worlds and the floating origin¶
Source: docs/design/huge-worlds.md §1.3, §1.4, §6, v0.2.0.md, v0.7.0.md.
Check |
Reference |
Tolerance |
Result |
Test |
|---|---|---|---|---|
Geodetic, ECEF and ENU frames |
GeographicLib |
round trip < 1e-6 m within 100 km |
met |
|
A world 500 km out |
the same world at the origin |
bit-identical |
poses bit-identical, lidar within 1.2e-10 m after 10 s (without the floating origin: 5e-7 m in pose, 8e-5 m in range) |
|
Rebase during a run |
the run never rebased |
— |
within 1e-9 m on MuJoCo, 1e-4 m on Jolt |
|
Robots 10 km apart |
one robot alone at the origin |
exact |
met |
|
Tile streaming |
deterministic selection; no entry into an unloaded tile |
equal state hashes |
met |
|
Terrain from an elevation model |
the raster’s height at each vertex’s geodetic point |
1e-3 m |
met |
|
Owed: a drone flight crossing two rebases with continuous MAVLink GNSS (huge-worlds §6, v0.4; no evidence found); memory budgets from profiles; the viewer’s frame time while tiles load; robots moving between regions while running.
Radiation¶
Source: domains/radiation.md. Not built: the release is after v0.7 (open question Q2) and the
owner placed it after 1.0. The domain exists as a contract with its zero substitute only. The
planned checks (§7) are the inverse-square law, slab shielding with NIST XCOM coefficients,
build-up factors (ANSI/ANS-6.4.3 values as reproduced in open references), field sampling, SEU
statistics and a Geant4 comparison within a factor of 2. None is implemented. Sim-vs-real: none.
Hydrodynamics¶
Source: domains/hydrodynamics.md. Post-1.0; only the contract and the zero substitute exist.
The planned checks (§7, “when built”) are hydrostatic equilibrium of a box, terminal velocity
under quadratic drag, turning circle and zig-zag of a documented vessel, and wave response from
RAO tables. None is implemented. Sim-vs-real: none.
Open items across domains¶
Q16 sim-vs-real data: hardware and publication terms still to be named. Until then the battery is the only model compared with measured data.
Published datasets owed: small-propeller data (UIUC, no licence stated), small-rotor ground effect, fog- and rain-chamber lidar data (DENSE), field measurements near power lines, ITU-R P.1238 parameters, an openly licensed MF tyre set, a published RCS reference, a pedestrian fundamental diagram.
Checks in a design table with no test found: lidar intensity, GNSS error statistics, sensor timing, vehicle energy balance.