Viewer¶
Status: draft for review; the stack is decided in rfcs/0006-native-viewer-stack.md.
lagrange_viewer is a native C++20 desktop program for Ubuntu 26.04. It draws with Vulkan through
gpu/hal/, builds its UI with Dear ImGui (docking branch), and talks to a simulation only
through the protocol (docs/design/protocol.md). It contains no web technology (invariant 13).
1. Stack (proposed, see RFC 0006)¶
Concern |
Choice |
Source |
Basis |
|---|---|---|---|
Window, input, Vulkan surface |
SDL3 |
apt |
Wayland by default with X11 fallback; per-window display scale; IME; the ImGui SDL3 backend lists multi-viewport and IME support, while the GLFW backend header lists parent-viewport support as missing (verification §9) |
UI |
Dear ImGui, docking branch, pinned commit |
|
The Ubuntu package is the master branch without docking (verification V1) |
Plots |
ImPlot v1.0 |
|
MIT, not in apt |
Schematic canvas |
imgui-node-editor behind |
|
MIT; last release 2023, so it sits behind an interface that an in-house canvas can replace (Q11) |
Gizmo |
ImGuizmo 1.10 |
|
MIT |
Font rasteriser |
FreeType through |
apt |
Better hinting for CJK at small sizes |
Fonts |
Noto Sans + Noto Sans JP + a monospace font, bundled |
|
SIL OFL (TODO(verify) the license files of the exact font builds bundled) |
Vulkan helpers |
volk (loader), Vulkan Memory Allocator |
apt |
Both are in Resolute |
Shaders |
Slang → SPIR-V at build time |
apt |
Shared with sensor kernels |
Protocol |
Protobuf C++ |
apt |
— |
UI tests |
Dear ImGui Test Engine, fetched in CI only |
not shipped |
Custom license (verification V13, Q12) |
2. Module layout and dependencies¶
viewer/
├── app/ main, CLI, Window (SDL3), Swapchain, FrameLoop, Settings
├── render/ FrameGraph, SceneRenderer, passes, GpuScene (mesh/material upload), Camera
├── client/ ViewerClient, FrameReader, ShmMap, SceneStore, SignalStore
├── panels/ Panel interface, registry, one directory per panel
├── layout/ dock layouts, presets, .ini persistence
└── assets/ fonts, icons, default layouts
Dependency rule inside the viewer: panels → client, render (read-only views);
render → gpu/hal, client (SceneStore); client → proto. client/ has no ImGui or Vulkan
includes, so it can be unit-tested and fuzzed without a display.
3. Window and surface layer¶
SDL_Init(SDL_INIT_VIDEO | SDL_INIT_GAMEPAD); window created withSDL_WINDOW_VULKAN | SDL_WINDOW_RESIZABLE | SDL_WINDOW_HIGH_PIXEL_DENSITY.The Vulkan instance is created by
gpu/hal/. The viewer passes the extension list fromSDL_Vulkan_GetInstanceExtensionsand creates the surface withSDL_Vulkan_CreateSurface.gpu/hal/never includes SDL headers; it receives aVkSurfaceKHR.Video driver selection is left to SDL (Wayland first, then X11).
--x11and--waylandsetSDL_HINT_VIDEO_DRIVERfor testing. CI runs the headless smoke test with theoffscreenordummySDL video driver and an offscreen HAL target (TODO(verify) which SDL3 driver gives a Vulkan surface without a compositor; otherwise the test renders without a swapchain).Swapchain: FIFO present mode by default (vsync),
MAILBOXwhen available and--low-latencyis set. Recreated onSDL_EVENT_WINDOW_PIXEL_SIZE_CHANGEDand onVK_ERROR_OUT_OF_DATE_KHR.When the window is minimised or occluded, the frame loop sleeps on
SDL_WaitEventTimeout(100 ms) and does not render, but the client worker keeps receiving so scopes do not lose data.
Wayland and X11 behaviour¶
Topic |
Wayland |
X11 / XWayland |
|---|---|---|
Window position |
Cannot be set or read by the client. The layout file stores size and maximised state only |
Position and size restored |
Decorations |
libdecor on GNOME (client-side); package dependency on |
Server-side |
Multi-viewport (torn-off panels) |
Off by default. With |
Available as an option; still off by default so behaviour is the same on both |
Fractional scaling |
|
Scale from |
IME (Japanese input) |
text-input protocol through SDL; |
IBus/Fcitx through SDL |
Clipboard, drag-and-drop of files |
SDL events ( |
same |
Session restore |
Layout comes from our own |
— |
The default is one fully docked window (CLAUDE.md §13.5). Every feature must be reachable in that mode.
4. Vulkan frame graph¶
flowchart LR
subgraph viewport["Viewport (offscreen, viewport panel size x scale)"]
depth["depth pre-pass"] --> opaque["opaque pass: meshes, PBR-lite"]
opaque --> overlay3d["overlay pass: grid, axes, contacts, force arrows, field slices"]
overlay3d --> select["selection outline + id buffer"]
select --> resolve["MSAA resolve -> viewport colour image"]
end
subgraph ui["UI"]
imgui["ImGui pass -> swapchain image"]
end
resolve -->|sampled as ImTextureID| imgui
sensor["sensor view images (from protocol)"] -->|uploaded textures| imgui
imgui --> present["present"]
The 3D scene is drawn into an offscreen colour image and shown in the viewport panel as an ImGui image. Its size follows the panel, so the scene is not rendered for pixels hidden by other panels.
Passes are described by a small
FrameGraph(pass name, attachments, read/write resources) so that later passes (shadow map, ray-traced ambient occlusion on Tier 2, thermal false colour) are additions, not rewrites.Realistic shading (
Shading::Realistic,RealisticPass,gpu/kernels/camera/realistic.slang; the default on a GPU, Lit stays for inspection and for software devices). In order: the sun’s depth (two cascades), the scene turned over the water’s surface at half size (what the water reflects), the scene in HDR (Cook-Torrance GGX from each material’s metallic and roughness, sky light by Lazarov’s split-sum, grain where a surface is rough), sky, water, see-through things, then ambient occlusion, bloom, ACES tone map and FXAA, with overlays drawn on top. A water prim is one withWaterAPI_1: Gerstner waves whose height follows the site’s wind, ripples, Fresnel reflection, depth-based colour, caustics, foam where it meets things. Limits: what the water looks like does not reach physics, which stays a flat surface, so things float level on the drawn waves and leave no wake; one water body is drawn as water, others as glass; the sun has a fixed place; no textures; about 15 ms a frame at 1080p on an Iris Xe. Camera sensors draw colour with the samegpu::RealisticRendereron a GPU (invariant 13); a software device keepsmesh.slang, and depth and instance ids always do. Rain and snow (falling,fallingKindin the shader): the run’s rain from EnvironmentState and its snow from the signalsenv/snow_rateandenv/snow_cover(the environment message has no snow; no protocol change) fall through the world as drops on a lattice that moves with the wind and their fall speed (rain at Atlas’s speed over Marshall–Palmer drops, as the aerodynamics domain has it; snow at 1 m/s), walked along each pixel’s ray up to what it sees, 22 m at most (18 m for snow). Drops are streaks of a 1/30 s exposure, flakes swaying dots; how many are drawn grows with the rate (a drawing choice: real rain has thousands of drops per m³). With Summer picked, falling rain brings the rainy season’s look and snow (or a third of the ground under it) the winter’s (effectiveSeason); a season picked by hand stays. Camera sensors do not see the falling drops yet (their fog veil and lens drops are the sensors domain’s).An id buffer (
R32_UINT, prim id) is rendered for the pixel under the cursor only (scissored 1×1 readback) for picking.MSAA 4× on Tier 1 and 2; off on lavapipe.
Frame pacing: one frame in flight by default to keep input latency low; two with
--throughput.
5. Threading and the lock-free hand-off¶
Thread |
Owns |
Blocks on |
|---|---|---|
UI thread (main) |
SDL events, all ImGui calls, all Vulkan submission, |
vsync only |
Client worker |
socket, frame reader, Protobuf decoding, shm mapping, |
socket |
Loader pool (2 threads) |
texture decode, mesh preprocessing (tangents, bounds), font atlas build |
job queue |
Hand-off design:
Scene state uses a triple buffer. The worker applies snapshots and deltas to a back copy of the transform array and an append-only change list, then publishes it by exchanging an atomic index.
FrameLoop::applyIncomingState()takes the latest published buffer with one atomic exchange. If the UI is slower than the server, intermediate states are skipped, not queued.Structural changes (prims added/removed, meshes, materials) travel through a single-producer single-consumer queue of
SceneEditrecords that the UI thread drains with a time budget of 2 ms per frame. Mesh uploads continue over several frames; a prim whose mesh is not uploaded yet draws its bounding box.Signals go into one
ScopeBufferper subscribed signal: a single-producer single-consumer ring that the worker fills and the UI reads without locks (section 7).Commands from panels go into a multi-producer queue (only the UI thread produces in practice) that the worker drains and sends. Results come back through a small SPSC queue and are matched by
request_id.No mutex is taken by the UI thread in the steady state.
tsanbuilds run the viewer’s integration test to check this.
// viewer/app/FrameLoop.hpp
namespace lagrange::viewer {
struct FrameStats {
double frameMs, uiBuildMs, viewportRenderMs, uiRenderMs, presentMs;
std::uint64_t frameIndex;
std::uint32_t sceneEditsApplied, signalSamplesRead;
};
class FrameLoop {
public:
FrameLoop(Window&, gpu::Device&, ViewerClient&, PanelRegistry&, layout::LayoutManager&);
void run(); // until the window closes or requestQuit()
void requestQuit();
bool runFrames(std::uint32_t count); // headless tests: render exactly N frames
const FrameStats& lastStats() const;
private:
void pollEvents(); // SDL3 events -> ImGui IO, resize, DPI change, file drop
void applyIncomingState(); // triple-buffer swap + budgeted SceneEdit drain (lock-free)
void buildUi(); // PanelRegistry::drawAll(); UI thread only
void renderViewport(gpu::CommandContext&); // 3D scene into the offscreen image
void renderUi(gpu::CommandContext&); // ImGui draw data; viewport shown as an ImGui image
void present();
void handleDisplayScaleChange(float newScale); // rebuild font atlas, rescale style
};
} // namespace lagrange::viewer
6. Panels¶
6.1 Interface and registry¶
// viewer/panels/Panel.hpp
namespace lagrange::viewer {
// Everything a panel may touch. Panels never see the socket or Vulkan objects directly.
struct PanelContext {
const SceneStore& scene; // read side; valid for the current frame
SignalStore& signals; // subscribe/unsubscribe, ScopeBuffer access
CommandSink& commands; // enqueue protocol commands; non-blocking
Selection& selection; // shared selection (prim ids)
const ServerStatusView& status; // sim time, run state, RTF, capabilities
const ProfileStateView& profile;
render::ViewportHandle viewport; // texture id + camera control for the 3D panel
float uiScale; // current display scale
double frameTimeSec;
};
class Panel {
public:
virtual ~Panel() = default;
virtual std::string_view id() const = 0; // stable, used in layout files: "scope"
virtual std::string_view title() const = 0; // shown in the tab; may be localised
virtual bool allowsMultipleInstances() const { return false; }
// Called once per frame while the panel is open. UI thread. Must not block or allocate in
// the steady state (CLAUDE.md §13.5).
virtual void draw(PanelContext&) = 0;
// Called when the panel becomes visible/hidden (e.g. its tab is not selected). A hidden
// scope keeps its subscriptions; a hidden sensor view drops them.
virtual void onVisibilityChanged(bool /*visible*/, PanelContext&) {}
// Persistent per-instance settings, stored in the layout file next to ImGui's own data.
virtual void saveSettings(SettingsWriter&) const {}
virtual void loadSettings(const SettingsReader&) {}
};
struct PanelDescriptor {
std::string_view id;
std::string_view menuPath; // "View/Scopes/Waveform scope"
std::unique_ptr<Panel> (*create)();
bool openByDefault;
};
class PanelRegistry {
public:
void add(const PanelDescriptor&);
std::span<const PanelDescriptor> descriptors() const;
Panel& open(std::string_view id); // creates another instance if allowed
void close(Panel&);
void drawAll(PanelContext&); // Begin/End + draw() for each open instance
void drawViewMenu();
void saveAll(SettingsWriter&) const;
void loadAll(const SettingsReader&);
};
#define LAGRANGE_REGISTER_PANEL(Type, id, menuPath, openByDefault) /* static registration */
} // namespace lagrange::viewer
Third-party panels are out of scope for v0.1. The registry is the place where a plugin mechanism would attach later.
6.2 v0.1 panel catalog¶
Each sketch shows the default content. All lists use ImGuiListClipper.
3D viewport (viewport)
┌ Viewport ───────────────────────────────────────────────────────────┐
│ [Orbit|Fly] [Select|Move|Rotate] [Grid] [Axes] [Contacts] [Forces] │
│ [Field: none ▾] [Shading: lit ▾] 60 fps 4 ms│
│ │
│ (offscreen scene image) │
│ │
│ x 12.340 m y -3.201 m z 0.000 m | 33.5902° N 130.4017° E │
└─────────────────────────────────────────────────────────────────────┘
Orbit (Alt+LMB or MMB), pan, zoom to cursor, fly (RMB + WASD), frame selection (F). Click selects
through the id buffer. The gizmo sends SetTransform and is disabled when the server lacks
can_edit_scene. The geodetic readout appears only for georeferenced stages.
Scene tree (scene_tree)
┌ Scene ──────────────────────────────┐
│ 🔍 filter… [types ▾]│
│ ▾ amr Xform 👁 │
│ ▸ base_link Xform 👁 │
│ ▸ left_wheel Xform 👁 │
│ ▾ power Scope │
│ battery Xform ⚡🌡 │
│ dc_48v Xform ⚡ │
│ ▸ sensors Scope │
└─────────────────────────────────────┘
Badges show applied schema families (electrical, thermal, EMI, sensor). Selection is shared with the viewport, inspector and schematic. A stage with 100 000 prims must scroll at 60 fps: the tree keeps a flattened visible-row index that is rebuilt only when expansion or filter changes.
Property inspector (inspector)
┌ Inspector: /amr/left_drive_motor ───────────────────────┐
│ ▾ Transform │
│ translate 0.000 0.250 0.080 m │
│ ▾ ElectricalMotorAPI (v1) │
│ windingResistance [ 0.120 ] Ω ● authored │
│ inductance [ 3.1e-4 ] H ● authored │
│ polePairs [ 4 ] │
│ resistanceTempCoeff [ 0.00393 ] 1/K ○ fallback │
│ powerPort → /amr/power/dc_48v │
│ ▾ ThermalAPI (v1) │
│ heatCapacity [ 450 ] J/K │
│ ▾ Live signals │
│ joint/left_wheel/torque 1.92 N·m [+ scope] │
│ motor/left_drive_motor/current_q 22.6 A [+ scope] │
└─────────────────────────────────────────────────────────┘
Attributes come from GetAttributes, with unit and documentation from the schema. Edits send
SetAttribute. “Live signals” lists catalog entries whose prim is the selection.
Domain toggle panel (domains)
┌ Domains ────────────────────────────────────────────────────────────┐
│ Profile: amr_warehouse_nav2.yaml* [Validate] [Apply] [Save…] │
│ ☑ rigid_body backend [mujoco ▾] dt [1 ms ] ▮▮▮▯▯ 31 % │
│ ☑ electrical fidelity [averaged ▾] dt [100 µs] ▮▮▯▯▯ 18 % │
│ ☑ thermal dt [100 ms] ▯▯▯▯▯ 0 % │
│ ☐ aerodynamics substitute [zero ▾] │
│ ☐ weather substitute [clear ▾] │
│ ☐ emi substitute [zero ▾] │
│ ☐ radiation substitute [zero ▾] │
│ ☑ sensors dt [5 ms] ▮▯▯▯▯ 9 % │
│ ⚠ /domains/electrical/dt: not a whole number of nanoseconds │
│ ▸ YAML preview │
└─────────────────────────────────────────────────────────────────────┘
The panel edits a YAML document in memory. It validates locally against the JSON Schema that the
server sent in ProfileState.json_schema, then with ValidateProfile for the checks a schema
cannot express. A disabled domain always has a substitute selected, so the panel cannot produce
an invalid profile. “Save…” writes the YAML file; the acceptance test loads the saved file with
the schema validator.
Timeline (timeline)
┌ Timeline ───────────────────────────────────────────────────────────┐
│ [⏮] [▶/⏸] [⏭ step] RTF target [1.00×] measured 0.98× t = 12.340 s│
│ ├──────────────●──────────────────────────────────────┤ 0 … 95.2 s │
│ markers: ▲ log errors ◆ profile changes │
└─────────────────────────────────────────────────────────────────────┘
The scrub bar is active when can_seek is true. Dragging sends Seek at most 30 times per
second.
Waveform scope (scope, multiple instances)
┌ Scope 1 ────────────────────────────────────────────────────────────┐
│ [+ signal] [⏸ freeze] window [200 ms ▾] [trigger: off ▾] [cursors] │
│ 30 A ┤ ╭╮ ╭╮ — motor/left/current_q A │
│ ┤╭╮ ╭╯╰╮ ╭─╯╰─╮ — motor/right/current_q A │
│ 0 A ┼╯╰──╯ ╰╯ ╰──── │
│ 52 V ┤────────╮ ╭────────── — bus/dc_48v/voltage V │
│ 48 V ┤ ╰──╯ │
│ 12.20 12.30 12.40 s Δt 3.1 ms Δy 4.2 A │
└─────────────────────────────────────────────────────────────────────┘
Signals with different units get separate stacked Y axes; signals with the same unit share one. Dropping a signal from the inspector or scene tree adds it. See section 7 for data handling.
Electrical schematic (schematic)
┌ Schematic ──────────────────────────────────────────────────────────┐
│ [auto layout] [show: current ▾] [scale: 30 A] │
│ ┌─────────┐ dc_48v 51.3 V ┌──────────────┐ │
│ │ battery ├──►──►──►──┬──►──►─┤ left_motor │ 22.6 A │
│ │ SoC 82% │ │ └──────────────┘ │
│ └─────────┘ ├──►──►─┤ right_motor │ 21.9 A │
│ └──►────┤ computer │ 0.9 A │
└─────────────────────────────────────────────────────────────────────┘
Nodes are prims with ports; edges are nets from SceneSnapshot.nets. Animated dashes move along
each edge at a speed proportional to the current signal, in the direction of its sign. Clicking a
node selects the prim everywhere. Node positions are stored in the layout file, keyed by prim
path; “auto layout” places sources left and loads right.
Field overlay controls (fields): lists Field3D signals, lets the user pick one, a slice
plane (axis and offset) or iso-level, a colour map and range. The viewport draws the slice. In
v0.1 the mock server supplies a sample field as a dense grid in a Blob.
Console/log (console): LogRecord stream with level and source filters, text search,
copy, and “pause on error”. A ring of 100 000 lines.
ROS 2 topic browser (ros_topics): table of topics from RosGraph (name, type, publishers,
subscribers, rate). Numeric fields of topics that the server exposes as signals can be dropped
into a scope. When has_ros_graph is false the panel says so.
Layout presets (menu, not a panel): “Default”, “Electrical debugging” (schematic + 2 scopes + inspector), “Navigation” (viewport + ROS topics + console), “Replay review” (viewport + timeline
scopes).
Replay comparison (design 08-replay: Compare, Sim vs real, Validation, History,
viewer/panels/src/ReplayCompare.cpp, viewer/client/src/CompareLog.cpp). The “real” log is
any second Lagrange recording: a real robot’s log converted into one, or another simulated run
(another engine, profile or parameter; the project has no hardware logs). Both files are read on
the viewer’s machine as an open format (MCAP, formats.md §7), as Foxglove or Python would; the
viewer links no simulation module. Each signal is averaged into bins of the resample rate
(default 100 Hz), aligned by the lag (±0.5 s) that maximises the normalised cross-correlation,
and scored by NRMSE over the real series’ range against a per-signal limit. Without a suite file
every scalar signal both logs share (except the recorder’s poses and signals constant in both)
is compared with a 5 % limit; a suite file names signals, components, limits and the method:
suite: amr_drive
method: {alignment: cross-correlation, resample_hz: 100, max_lag_s: 0.5, window: [0, 10]}
signals:
- {sim: motor/left_drive_motor/current_q, real: motor/left_drive_motor/current_q, limit: 0.05}
- {sim: body/base_link/twist, component: 0, limit: 0.05}
Driving phases (stopped, accelerate, cruise, brake) come from the replay’s first body twist.
lagrange_viewer --compare OTHER.mcap [--suite FILE.yaml] starts a comparison; Export report
writes JSON to <config>/validation/. Not built yet, and why:
A replay served from another machine cannot be compared: the viewer would need the server to read the second file. Proposed additive protocol fields (owner approval, 1.x additive rule):
OpenRecording.compare_path = 2(string: open a second recording read-only beside the replay),RecordingInfo.compare = 11(RecordingInfo: its facts), and the compare log’s signals in the normalSignalCatalogunder acompare/name prefix withSignalInfo.compare = 11(bool), so the panels subscribe to both through the protocol.Re-simulate from here… (Source strip) is shown disabled: a live run seeded from a recorded time needs the server to start from that state, which a recording does not hold in full (it holds signals, not module states). Proposed:
Command.resimulate = 35withmessage Resimulate { SimTime from = 1; string profile_yaml = 2; }, the server starting a live run of the recording’s stage with root poses and joint positions from the recorded signals atfrom(as spawning carries state over), answered byCommandResult.“Overlay in 3D” (the sim and real robots drawn together) and probable-cause text in the failure card; the card states the largest error, its time and the two values instead.
6.3 Robot builder (RFC 0016)¶
RFC 0016 (rfcs/0016-robot-builder.md, accepted, with Addendum A) is the Builder workspace:
build a robot in the viewer, test it in a test server of the viewer’s own, save it as URDF. The
user page is docs/guide/builder.md; the screen’s design docs/design/ui/12-builder/; the work
plan docs/plan/builder-phase1-plan.md.
Where it lives.
viewer/builder/(lagrange_viewer_builder): the document (links, shapes, joints, sensors), edits and the undo history, mass properties (primitives, closed meshes by Mirtich’s volume integrals), validation, the URDF reader and writer (deterministic, byte for byte:viewer/test/builder/golden/), the sensor catalog loader (modules/sensors/catalog/sensor_catalog.yaml, installed with the viewer), gizmo maths, collision choices and the split, and the test server. It links no simulation module (invariant 14); meshes are read bylagrange_meshes, as the server reads them, linked privately (with it thelagrange_corelibrary), so no builder header and no panel sees core. The panels areviewer/panels/src/Builder*.cpp;viewer/app/src/BuilderHost.cppruns the child processes (test server,lagrange_import,lagrange check,xacro,bake_urdf), each withposix_spawnand polled every frame, never blocking the UI thread.Modes. Build draws only the document’s robot on the Lit pipeline:
SceneAdapterturns the document into aSceneFramefor the viewer’s ownSceneRenderer, not from the protocol. Test writes the URDF to the runtime folder ($XDG_RUNTIME_DIR/lagrange/builder/<pid>/), startslagrange_server --robot F --world none --joint-control --listen unix:…, and points the client at it; Back sends SIGTERM (SIGKILL after 3 s) and returns to the endpoint the viewer had before, or to none.Cost rule (R7). While the Builder is in front,
ViewerClient::setQuiet(true)unsubscribes every signal and turns sensor frames, field slices, the ROS graph and world state off with existing messages (no protocol change); leaving sends again what the panels want. The headless report countssensor_frames,field_messages,world_messages,ros_graphsandstreamed_while_quiet(any of those or a signal block more than 1 s after going quiet);test_builder_acceptance.pychecks it is 0 against a real server.Flags.
--workspace builder,--builder-open FILE(.urdf, or.xacroby expansion),--builder-save FILE(headless: the document at the end; exit 5 when it cannot be written), Ctrl B (also on the launcher, without a server). The headless report has abuilderobject andwindows(the windows in front), which the input tests read.Measured (2026-11-15,
LAGRANGE_VIEWER_PROFILE=1, 1600 × 1000, the sample rover open, 300 frames, a shared laptop): llvmpipe frame 17.9 / 20.9 ms median / p95 (functional only, as §11 says for lavapipe); RTX 4060 Laptop 1.06 / 1.30 ms, and 1.17 / 2.44 ms while zooming the view every other frame, inside the 16.6 ms budget.Not built yet: Test into a sample world or a world file, Into the connected run (§7.2, phase 1b), joint sliders driving a test (§7.3), the overlay’s motors, battery and sensors (phase 3), USD and SDFormat (phase 4).
7. Scopes: ring buffers and decimation¶
Requirements: at least 16 signals at 10 kHz without dropped samples (CLAUDE.md §13.4), smooth drawing of windows from 1 ms to 10 min.
// viewer/client/ScopeBuffer.hpp
namespace lagrange::viewer {
// Single-producer (client worker) single-consumer (UI thread) ring of uniformly sampled values
// for one signal component. Lock-free: one atomic write cursor, one atomic generation.
class ScopeBuffer {
public:
// capacity is rounded up to a power of two. Default: 2^20 samples (104 s at 10 kHz).
ScopeBuffer(std::size_t capacitySamples, std::int64_t dtNs);
// --- producer side (worker thread) ---------------------------------------------------
// Appends a run. If firstIndex is not contiguous with the last sample, a gap is recorded.
void push(std::uint64_t firstIndex, std::int64_t t0Ns, std::span<const double> samples);
void reset(std::int64_t dtNs); // catalog generation changed or seek
// --- consumer side (UI thread) -------------------------------------------------------
struct Column { float min, max; }; // one per horizontal pixel
// Fills `out` (size = pixel columns) with min/max over each column's time span.
// Uses the min/max pyramid, so cost is O(columns · log(samplesPerColumn)), not O(samples).
// Returns the number of columns that contain at least one sample.
std::size_t decimate(std::int64_t tBeginNs, std::int64_t tEndNs, std::span<Column> out) const;
// Raw access when zoomed in (fewer samples than pixels). Up to two spans (ring wrap).
struct RawView { std::span<const double> first, second; std::int64_t t0Ns; std::int64_t dtNs; };
RawView raw(std::int64_t tBeginNs, std::int64_t tEndNs) const;
std::int64_t newestTimeNs() const;
std::int64_t oldestTimeNs() const;
std::uint64_t droppedSamples() const; // from index gaps; shown in the status bar
std::span<const std::int64_t> gapTimesNs() const; // drawn as markers
double valueAt(std::int64_t tNs) const; // for cursors
private:
// samples_: ring of f64. pyramid_[k]: ring of {min,max} over 4^(k+1) samples, k = 0..7.
// Pyramid levels are completed by the producer as blocks fill; the consumer reads only
// completed blocks plus the raw tail.
};
} // namespace lagrange::viewer
Notes:
Memory: 2²⁰ samples × 8 bytes = 8 MiB per component, plus about 1/3 more for the pyramid at
floatprecision. Sixteen signals use about 170 MiB. The capacity is a setting.Drawing: when a pixel column covers more than two samples, the scope draws a vertical min/max bar per column (
ImPlotshaded/bars item fed bydecimate). Otherwise it draws a polyline fromraw. This is the “min/max per pixel” decimation in CLAUDE.md §13.1.Server-side decimation: for windows longer than the ring or over a slow link, the scope subscribes with
MODE_MINMAX. Locally it always usesMODE_FULLso that zooming in after the fact shows real samples.Trigger (rising/falling edge on one signal with level and hold-off) is evaluated on the worker thread when samples arrive, so no edge is missed between frames.
Non-uniform signals (event-driven) use a second buffer type with explicit timestamps and a simpler stride decimation.
8. Protocol client¶
// viewer/client/ViewerClient.hpp
namespace lagrange::viewer {
struct Endpoint {
enum class Kind { UnixSocket, Tcp } kind;
std::string address; // socket path or host
std::uint16_t port{0};
};
enum class ConnectionState { Disconnected, Connecting, Handshaking, Ready, Refused, Lost };
struct ClientOptions {
bool useSharedMemory{true}; // only effective on Unix sockets
std::chrono::milliseconds reconnectDelay{1000};
std::size_t scopeCapacitySamples{1u << 20};
};
class ViewerClient {
public:
explicit ViewerClient(ClientOptions);
~ViewerClient(); // joins the worker
void connect(Endpoint); // returns immediately; worker connects
void disconnect();
ConnectionState state() const; // atomic
std::string refusalReason() const; // set when state() == Refused
// --- read side, UI thread only ---------------------------------------------------------
// Swaps in the newest scene state and drains structural edits within the time budget.
void pump(std::chrono::microseconds editBudget);
const SceneStore& scene() const;
SignalStore& signals();
const ServerStatusView& status() const;
const ProfileStateView& profile() const;
bool pollLog(LogRecordView& out); // SPSC, call until false
bool pollResult(CommandResultView& out);
// --- command side, UI thread only; never blocks -------------------------------------
std::uint64_t send(CommandVariant command); // returns request_id
CommandSink& sink();
struct Stats { std::uint64_t bytesIn, framesIn, decodeErrors, snapshots, deltas, resyncs, shmPayloads; };
Stats stats() const;
private:
void workerMain(); // poll(), read frames, decode, publish
};
// Subscription bookkeeping. Subscriptions are by name, so they survive catalog generations.
class SignalStore {
public:
struct Handle { std::uint32_t slot; };
Handle subscribe(std::string_view signalName, std::uint32_t component = 0);
void unsubscribe(Handle);
const ScopeBuffer* buffer(Handle) const; // nullptr until the signal exists
std::span<const SignalInfoView> catalog() const;
std::uint64_t catalogGeneration() const;
double latest(std::string_view signalName, std::uint32_t component = 0) const; // inspector
};
} // namespace lagrange::viewer
Shared-memory handling lives in client/ShmMap: it receives the file descriptors from the
handshake, maps the regions read-only, and resolves a ShmRef to a span after checking the
slot generation before and after the copy (protocol §6). Texture and mesh payloads are copied
once, directly into a staging buffer that gpu/hal/ uploads.
9. Layouts¶
ImGui’s docking state is saved to
~/.config/lagrange/viewer/layout.ini(XDG config directory). Our panel settings are stored in the same file through an ImGui settings handler, so one file describes a layout.Presets are
.inifiles inshare/lagrange_viewer/layouts/. “Reset layout” copies a preset over the user file. Users can save their own presets by name.Layout files store a
ui_scale-independent description (ImGui stores dock sizes as ratios within the dock space), so a layout saved on a HiDPI laptop opens correctly on a 1× monitor.A layout that references an unknown panel id (older or newer viewer) ignores that entry.
10. HiDPI and fonts¶
Scale source:
SDL_GetWindowDisplayScale. OnSDL_EVENT_WINDOW_DISPLAY_SCALE_CHANGEDthe viewer rebuilds the font atlas at the new pixel size and rescales the style from a pristine copy (ImGuiStyle::ScaleAllSizesapplied to the unscaled style, never cumulatively).Base font size 15 px at scale 1.0; user setting 0.8×–1.5× on top of the display scale.
Fonts: Latin UI font, Japanese font merged into the same ImFont, monospace font for the console and numeric fields, and an icon font.
Japanese glyphs: with the ImGui 1.92+ dynamic font atlas, glyphs are rasterised on demand, so the whole Jōyō set does not have to be baked at startup [Medium, from ImGui 1.92 release notes knowledge; TODO(verify) on the pinned docking commit]. If the pinned version lacks it, the fallback is
GetGlyphRangesJapanese()baked at startup, which costs atlas memory and start-up time and must be measured against the 5 s cold-start budget.Prim names, file paths and log text are UTF-8 end to end. A test stage with Japanese prim names (
/ロボット/左車輪) is part of the viewer tests.UI strings are English in v0.1 but go through a
tr("…")lookup from the start so that a Japanese translation is a data file, not a refactor.
11. Performance budgets and how they are measured¶
Budget (CLAUDE.md §13.4) |
Measurement |
Where |
|---|---|---|
≤ 16.6 ms frame on an RTX-class GPU, sample AMR scene, 8 scopes |
|
GPU runner, nightly |
≥ 16 signals at 10 kHz without drops |
|
PR (lavapipe) and GPU runner |
Cold start to first frame with a sample URDF < 5 s (warm disk) |
timestamp from |
GPU runner |
Runs on Tier 1 |
Smoke test on an integrated-GPU runner or with |
nightly |
Internal targets that support them:
Stage |
Target |
|---|---|
|
≤ 0.5 ms typical, 2 ms cap |
|
≤ 3 ms |
Scope draw, 8 scopes × 4 signals, 1000 px wide |
≤ 1.5 ms total |
Viewport render, 200 meshes, 1M triangles |
≤ 4 ms on RTX-class |
Worker decode of one 25 kB |
≤ 0.1 ms |
Steady-state heap allocations per frame on the UI thread |
0 (checked with an allocation counter in debug builds) |
On lavapipe the frame-time check is relaxed to a functional check (frames complete, golden image matches). Performance is asserted only on real GPUs.
What keeps a frame cheap (as built):
The 3D view is drawn again only when what it shows changed: the scene frame, the camera, the viewport size, the render options or an overlay (
Ui::renderScene,ViewKey). Paused, it is drawn once; running, at the server’s scene rate, not the display’s. Airflow particles, which move every frame, always redraw.Realistic: the opaque meshes’ depth first, then shading with no depth writes (each pixel is shaded once); meshes whose bounding ball is outside the view, or outside a shadow cascade’s box, are not drawn there (bounds from
MeshSet, so camera sensors cull too). Raster shadings cull to the view the same way, with bounds the client computes from the vertices it received.A stage’s meshes are uploaded in one vertex and one index buffer (two transfers, not two per mesh); lidar overlays and thermal cell faces are rebuilt only when their data changed; ImGui’s vertex buffers start at 4 MB so that a growing scope does not reallocate them every frame.
LAGRANGE_VIEWER_PROFILE=1 lagrange_viewer ... prints the median and 95th percentile of each
stage (UI build, command recording, GPU wait, present, whole frame) to stderr, at the end of a
headless run and every five seconds with a window, and how often the 3D view was drawn.
12. Tests¶
Unit (no display):
ScopeBuffer(wrap, pyramid against brute force, gaps),SceneStore(snapshot + delta sequences, resync), frame reader (fuzzed), profile YAML generation.Headless integration (PR, lavapipe): start the mock server with a sample recording, start the viewer with
--headless --frames 300 --layout default, compare the viewport image against a golden image (per-pixel tolerance; goldens are produced on lavapipe only), check that the domain panel’s saved YAML validates, check zero dropped samples.UI automation: ImGui Test Engine scripts for panel interactions (toggle a domain, add a signal to a scope, select in tree → inspector shows it). Compiled only when
LAGRANGE_VIEWER_UI_TESTS=ON; the engine is fetched at configure time in CI and is never part of an installed package (Q12).Sanitizers: ASan+UBSan and TSan runs of the headless integration test.
Manual matrix per release: GNOME Wayland, GNOME X11, KDE Wayland; 1× and fractional scale; NVIDIA proprietary driver and Mesa (Intel or AMD).
13. Relationship to RViz2 and other tools¶
RViz2 on Lyrical renders with OGRE 1.12 and OpenGL (verification V15), so no rendering code can be shared. The viewer does not host RViz display plugins. Interoperation is through data:
The ROS 2 bridge publishes standard messages (
tf2_msgs/TFMessage,sensor_msgs/JointState,sensor_msgs/Image,sensor_msgs/PointCloud2,sensor_msgs/LaserScan,nav_msgs/Odometry,visualization_msgs/MarkerArray), so RViz2, Foxglove (ros-lyrical-foxglove-bridgeexists) and PlotJuggler work unchanged next to our viewer.Our recordings are MCAP, which Foxglove and
ros2 bagtooling open.What the viewer adds over those tools is what they do not have: domain toggles, the unit-aware signal catalog, 10 kHz scopes fed without ROS serialisation, the electrical schematic, field overlays, and scene editing against the simulation.
Seasons and procedural surfaces¶
surfacePatternOf takes a surface from a word of a material’s name (brick, asphalt, pavement,
plaster, concrete, lawn, leaves, road_paint, window_glass, bark, roof_felt). Brick, asphalt, pavement,
plaster, concrete, lawn, bark and roofing use photographs (gpu/assets_textures, CC0), tiled by
world position and tinted to the material’s own colour, so meshes need no texture coordinates; a
viewer without the files, and the other kinds, use shader patterns. Season (--season, the Shading
menu) changes the sun, sky, haze, surfaces, puddles and falling rain or snow; all by eye and drawn
only. The effect of rain and snow on sensors and grip comes from the weather domain, which
lagrange --season sets. Water does not freeze. Camera sensors share all of this through
gpu::RealisticRenderer (--camera-look, --camera-season).