UI↔server protocol¶
Status: draft for review; decided in rfcs/0003-ui-server-protocol.md. This is the only channel
between lagrange_viewer and a simulation (invariant 14). The mock server and the real server
implement the same protocol; the viewer cannot tell them apart except through
ServerHello.server_kind.
1. Decisions proposed¶
Topic |
Proposal |
Basis |
|---|---|---|
Encoding |
Protobuf, proto3 syntax |
|
Bulk numeric data |
Raw little-endian arrays inside |
Avoids per-sample varint cost for 10 kHz signals and large transform sets |
Transport, same machine |
Unix domain stream socket |
Needed anyway to pass shared-memory file descriptors ( |
Transport, other machine |
TCP |
Same framing |
Shared memory |
|
Linux-only by design; hidden behind |
Zenoh |
Not in v0.1. Reserved as a later transport behind the same |
|
Security |
Unix socket: file mode 0600. TCP: bound to 127.0.0.1 unless |
Stated in |
2. Framing¶
Every message on the stream is:
+----------------+----------------------------+
| length: u32 LE | Envelope (protobuf), length bytes |
+----------------+----------------------------+
Maximum frame size: 64 MiB. Larger payloads (meshes, textures, point clouds) are chunked with
Blobmessages or sent through shared memory.Until the server has accepted its
ClientHello, a client may send frames of at most 64 KiB (proto::kPreHandshakeMaxFrameSize; RFC 0014 §1). A larger header closes the connection with aServerHellowhoserefusalsays why. Frames pipelined behind theClientHelloare read under the normal 64 MiB limit. What the server sends keeps the 64 MiB limit from the start.The first frame in each direction is a handshake (
ClientHello, thenServerHello). Any other first frame, and one whose payload does not parse, closes the connection, with aServerHellorefusal (“the first message must be ClientHello”).Before the handshake the server also limits connections (RFC 0014 §2 to §4): at most 16 waiting for their
ClientHelloand 32 open in all, at most 8 new TCP connections in a burst and then 2 a second from one address (IPv6: per /64), and 10 s from accept to theClientHello(TLS included). Over a limit a connection is sent aServerHellorefusal (best effort; not over TLS before its handshake) and closed.lagrange_server--max-connections N,--max-pending N,--handshake-timeout S,--peer-rate BURST,PER_SECONDchange them; the 64 KiB cap is a constant.ServerStats.viewerscounts greeted sessions only.Default endpoints: Unix socket
$XDG_RUNTIME_DIR/lagrange/<instance>.sock(instance defaults todefault); TCP port47610(a project default, configurable; not an IANA registration).
3. Message catalog¶
Direction |
Message |
When |
Size class |
|---|---|---|---|
C→S |
|
first frame |
small |
S→C |
|
reply |
small |
S→C |
|
10 Hz and on change |
small |
S→C |
|
after hello and when wiring changes |
medium |
C→S |
|
scopes opened/closed |
small |
S→C |
|
per subscription, batched (default every 20 ms) |
medium |
S→C |
|
on request and after a delta out of sequence |
large (chunked/shm) |
S→C |
|
at viewport rate (default 60 Hz) |
medium |
S→C |
|
chunks of meshes, textures, images |
large |
C→S |
|
user actions |
small |
S→C |
|
one per command |
small |
S→C |
|
after hello and on change |
small |
S→C |
|
console output |
small |
S→C |
|
1 Hz when the ROS panel is open |
medium |
S→C |
|
subscribed sensor views (image, point cloud) |
large (shm when local) |
S→C |
|
after hello and when another run is loaded |
medium |
S→C |
|
when something happens in a live run |
small |
S→C |
|
1 Hz |
small |
S→C |
|
after hello and when another recording is opened (replay) |
small |
both |
|
every 2 s; timeout 10 s |
small |
The viewer does not link rclcpp. The ROS 2 topic browser is filled from RosGraph, which the
server’s ROS bridge produces. The mock server fills it from the MCAP channel list.
4. Draft .proto files¶
Layout: proto/lagrange/viewer/v1/*.proto, package lagrange.viewer.v1. buf.yaml is in
docs/plan/repo-tooling.md.
4.1 common.proto¶
syntax = "proto3";
package lagrange.viewer.v1;
// Simulation time, integer nanoseconds since run start.
message SimTime { int64 ns = 1; }
// SI dimension exponents + scale + kind. Mirrors lagrange::Unit.
message Unit {
sint32 length = 1;
sint32 mass = 2;
sint32 time = 3;
sint32 current = 4;
sint32 temperature = 5;
sint32 amount = 6;
sint32 luminous = 7;
sint32 angle = 8;
double scale = 9; // 1.0 for SI
uint32 kind = 10; // lagrange::QuantityKind
string display = 11; // "N·m", for labels only
}
enum SignalType {
SIGNAL_TYPE_UNSPECIFIED = 0;
SIGNAL_TYPE_SCALAR = 1;
SIGNAL_TYPE_VEC3 = 2;
SIGNAL_TYPE_QUAT = 3;
SIGNAL_TYPE_POSE = 4;
SIGNAL_TYPE_WRENCH = 5;
SIGNAL_TYPE_SCALAR_ARRAY = 6;
SIGNAL_TYPE_BOOL = 7;
SIGNAL_TYPE_INT = 8;
SIGNAL_TYPE_BYTES = 9;
SIGNAL_TYPE_FIELD3D = 10;
SIGNAL_TYPE_IMAGE = 11;
SIGNAL_TYPE_POINT_CLOUD = 12;
}
// A payload that is either inline, chunked through Blob messages, or in shared memory.
message DataRef {
oneof location {
bytes inline = 1;
uint64 blob_id = 2; // assembled from Blob chunks
ShmRef shm = 3;
}
uint64 size = 4; // total bytes
fixed64 xxh3 = 5; // content hash; also the cache key for meshes and textures
}
message ShmRef {
uint32 region = 1; // index announced in ServerHello / ShmRegionAdded
uint64 offset = 2;
uint64 size = 3;
uint64 generation = 4; // must equal the slot header's generation when read
}
message Blob {
uint64 blob_id = 1;
uint64 offset = 2;
bytes data = 3; // <= 4 MiB per chunk
bool last = 4;
}
4.2 session.proto¶
syntax = "proto3";
package lagrange.viewer.v1;
import "lagrange/viewer/v1/common.proto";
message ProtocolVersion { uint32 major = 1; uint32 minor = 2; }
message ClientHello {
ProtocolVersion version = 1;
string client_name = 2; // "lagrange_viewer 0.1.0"
bool wants_shm = 3; // only honoured on Unix sockets
repeated string compression = 4; // "zstd"; empty = none
string locale = 5; // "ja_JP", for server-side message text (later)
string auth_token = 6; // v1.0: the token a TCP server requires (security.md)
}
enum ServerKind {
SERVER_KIND_UNSPECIFIED = 0;
SERVER_KIND_SIM = 1;
SERVER_KIND_MOCK = 2;
SERVER_KIND_REPLAY = 3;
}
message ServerHello {
ProtocolVersion version = 1;
ServerKind server_kind = 2;
string server_name = 3;
uint64 session_id = 4;
bool shm_enabled = 5; // file descriptors follow as ancillary data
repeated ShmRegion shm_regions = 6;
string compression = 7; // chosen, or empty
Capabilities capabilities = 8;
string refusal = 9; // non-empty = connection refused, human-readable reason
}
message ShmRegion { uint32 region = 1; uint64 size = 2; string purpose = 3; }
message Capabilities {
bool can_seek = 1; // replay/mock
bool can_step = 2;
bool can_edit_scene = 3;
bool can_apply_profile = 4;
bool can_import = 5; // URDF/SDF/MJCF import through the server
uint32 tier = 6; // 0, 1, 2
bool has_ros_graph = 7;
}
enum RunState {
RUN_STATE_UNSPECIFIED = 0;
RUN_STATE_STOPPED = 1;
RUN_STATE_PAUSED = 2;
RUN_STATE_RUNNING = 3;
RUN_STATE_ERROR = 4;
}
message ServerStatus {
SimTime sim_time = 1;
RunState state = 2;
double real_time_factor = 3; // measured
double real_time_factor_target = 4;
SimTime replay_begin = 5; // replay only
SimTime replay_end = 6;
uint64 dropped_deltas = 7;
uint64 dropped_signal_blocks = 8;
string error = 9;
}
message Ping { uint64 nonce = 1; int64 client_monotonic_ns = 2; }
message Pong { uint64 nonce = 1; int64 client_monotonic_ns = 2; SimTime sim_time = 3; }
enum LogLevel { LOG_LEVEL_UNSPECIFIED = 0; LOG_LEVEL_DEBUG = 1; LOG_LEVEL_INFO = 2; LOG_LEVEL_WARN = 3; LOG_LEVEL_ERROR = 4; }
message LogRecord {
SimTime sim_time = 1;
int64 wall_unix_ns = 2;
LogLevel level = 3;
string source = 4; // "scheduler", "electrical/averaged_network"
string text = 5;
}
message RosGraph {
message Topic { string name = 1; string type = 2; uint32 publishers = 3; uint32 subscribers = 4; double rate_hz = 5; }
repeated Topic topics = 1;
repeated string nodes = 2;
string rmw = 3;
}
4.3 scene.proto¶
syntax = "proto3";
package lagrange.viewer.v1;
import "lagrange/viewer/v1/common.proto";
// Positions are double. The snapshot states the origin of the local frame so that a viewer can
// show geodetic coordinates and survive a floating-origin rebase.
message LocalFrame {
double ecef_x = 1; double ecef_y = 2; double ecef_z = 3; // m; all zero if not georeferenced
repeated double enu_from_ecef = 4; // 9 values, row-major
uint64 rebase_count = 5; // increments at each rebase
repeated double region_enu = 6; // 3 values: world ENU of the region-local origin, m (v0.7)
}
message Prim {
uint32 id = 1; // dense, stable for the session
uint32 parent = 2; // 0xFFFFFFFF for roots
string name = 3;
string path = 4; // full USD path
string type_name = 5; // "Xform", "Mesh", "PhysicsRevoluteJoint"
repeated string api_schemas = 6;
uint32 mesh = 7; // index into SceneSnapshot.meshes, or 0xFFFFFFFF
uint32 material = 8;
bool visible = 9;
uint32 child_count = 10; // lets the scene tree virtualize without the whole list
}
message Attribute {
string name = 1; // "elec:windingResistance"
oneof value {
double real = 2;
int64 integer = 3;
bool boolean = 4;
string token = 5;
string asset = 6;
RealArray reals = 7;
Targets targets = 8; // relationship
}
Unit unit = 9;
bool editable = 10;
bool authored = 11; // false = schema fallback
string doc = 12;
}
message RealArray { repeated double v = 1; }
message Targets { repeated uint32 prims = 1; }
message PrimAttributes { uint32 prim = 1; repeated Attribute attributes = 2; }
message Mesh {
uint32 id = 1;
uint32 vertex_count = 2;
uint32 index_count = 3;
DataRef positions = 4; // float32 x3, mesh-local
DataRef normals = 5; // float32 x3
DataRef uvs = 6; // float32 x2, optional
DataRef indices = 7; // uint32
repeated float aabb = 8; // min xyz, max xyz
}
message Material {
uint32 id = 1;
string name = 2;
repeated float base_color = 3; // rgba
float metallic = 4;
float roughness = 5;
repeated float emissive = 6;
DataRef base_color_texture = 7; // PNG/JPEG/KTX2 bytes, optional
string base_color_texture_mime = 8;
}
// Electrical topology for the schematic panel. Derived from PortAPI / ElectricalBusAPI prims.
message ElectricalNet {
uint32 id = 1;
string name = 2; // "dc_48v"
uint32 bus_prim = 3;
repeated NetTerminal terminals = 4;
string voltage_signal = 5; // signal name, empty if none
}
message NetTerminal { uint32 prim = 1; string port = 2; string current_signal = 3; }
message SceneSnapshot {
uint64 sequence = 1; // deltas apply on top of this
SimTime sim_time = 2;
LocalFrame frame = 3;
repeated Prim prims = 4;
DataRef transforms = 5; // prim_count x (3 f64 position + 4 f32 quat + 3 f32 scale), prim id order
repeated Mesh meshes = 6;
repeated Material materials = 7;
repeated ElectricalNet nets = 8;
uint64 topology_hash = 9;
string stage_path = 10;
// Attributes are not included. The inspector requests them per prim (GetAttributes command).
}
message SceneDelta {
uint64 sequence = 1; // snapshot.sequence + n
uint64 base_sequence = 2; // must equal the client's current sequence, else resync
SimTime sim_time = 3;
// Changed transforms: parallel arrays.
repeated uint32 transform_prims = 4 [packed = true];
bytes transform_data = 5; // per entry: 3 f64 + 4 f32 (scale never changes in a delta)
repeated PrimAttributes attribute_changes = 6;
repeated Prim added = 7;
repeated uint32 removed = 8 [packed = true];
LocalFrame rebased_frame = 9; // set only when the floating origin moved
repeated Overlay overlays = 10; // contact points, force arrows
}
message Overlay {
enum Kind { KIND_UNSPECIFIED = 0; KIND_CONTACT_POINT = 1; KIND_FORCE_ARROW = 2; KIND_TORQUE_ARROW = 3; KIND_PATH = 4; }
Kind kind = 1;
bytes data = 2; // kind-specific packed f64: point(3)+normal(3)+magnitude(1) ...
uint32 count = 3;
}
message SensorFrame {
string sensor = 1; // "sensor/front_lidar"
SimTime sim_time = 2;
oneof payload { ImageData image = 3; PointCloudData points = 4; }
}
message ImageData { uint32 width = 1; uint32 height = 2; string encoding = 3; DataRef data = 4; } // "rgba8", "r32f"
message PointCloudData { uint32 count = 1; uint32 stride = 2; repeated string fields = 3; DataRef data = 4; } // "x","y","z","intensity"
4.4 signals.proto¶
syntax = "proto3";
package lagrange.viewer.v1;
import "lagrange/viewer/v1/common.proto";
message SignalInfo {
uint32 id = 1; // dense, stable until the catalog generation changes
string name = 2; // "joint/left_wheel/torque"
SignalType type = 3;
Unit unit = 4;
uint32 components = 5; // f64 components per sample
int64 native_dt_ns = 6; // writer's step; 0 for event-driven
string writer = 7; // "electrical/averaged_network"
bool substituted = 8; // written by a substitute
uint32 prim = 9; // related prim or 0xFFFFFFFF
repeated string component_names = 10; // "x","y","z"
}
message SignalCatalog {
uint64 generation = 1; // changes when wiring changes (profile applied, stage reloaded)
repeated SignalInfo signals = 2;
}
message Subscribe {
uint64 catalog_generation = 1;
repeated SignalSubscription signals = 2;
}
message SignalSubscription {
uint32 signal = 1;
// 1 = every sample. n > 1 = server sends min/max pairs over n samples (decimation="minmax")
// or every n-th sample (decimation="stride").
uint32 decimation = 2;
enum Mode { MODE_UNSPECIFIED = 0; MODE_FULL = 1; MODE_MINMAX = 2; MODE_STRIDE = 3; }
Mode mode = 3;
uint32 component_mask = 4; // bit i = component i; 0 = all
}
message Unsubscribe { repeated uint32 signals = 1; }
enum SampleEncoding {
SAMPLE_ENCODING_UNSPECIFIED = 0;
SAMPLE_ENCODING_F64_LE = 1;
SAMPLE_ENCODING_F32_LE = 2; // allowed only for MINMAX display streams
}
// A run of consecutive samples of one signal.
message SignalRun {
uint32 signal = 1;
SimTime t0 = 2; // time of the first sample
int64 dt_ns = 3; // uniform spacing; 0 = see timestamps
uint32 sample_count = 4;
uint32 components = 5;
SampleEncoding encoding = 6;
bytes samples = 7; // sample_count x components values; MINMAX: x2 (min then max)
bytes timestamps_ns = 8; // int64 LE per sample, only when dt_ns == 0
uint64 first_index = 9; // writer step index of the first sample; gaps reveal drops
SignalSubscription.Mode mode = 10;
}
// All runs produced since the previous block (default: 20 ms of simulation per block).
message SignalBlock {
uint64 catalog_generation = 1;
repeated SignalRun runs = 2;
}
Sizing check for the §13.4 budget: 16 scalar signals at 10 kHz in MODE_FULL, F64_LE,
blocks every 20 ms → 16 × 200 samples × 8 bytes = 25.6 kB per block, 1.28 MB/s. This is small for
a local socket and acceptable on a LAN.
4.5 command.proto¶
syntax = "proto3";
package lagrange.viewer.v1;
import "lagrange/viewer/v1/common.proto";
import "lagrange/viewer/v1/scene.proto";
message Command {
uint64 request_id = 1; // echoed in CommandResult
oneof command {
Play play = 10;
Pause pause = 11;
Step step = 12;
Seek seek = 13;
SetRealTimeFactor set_rtf = 14;
Reset reset = 15;
RequestSnapshot request_snapshot = 20;
GetAttributes get_attributes = 21;
SetAttribute set_attribute = 22;
SetTransform set_transform = 23;
SetVisibility set_visibility = 24;
LoadStage load_stage = 30;
ImportRobot import_robot = 31;
OpenRecording open_recording = 32;
RemoveRobot remove_robot = 33; // added after 1.0 (RFC 0016 §7.2)
CommandJoint command_joint = 34; // added after 1.0 (RFC 0016 §7.3)
ValidateProfile validate_profile = 40;
ApplyProfile apply_profile = 41;
GetProfile get_profile = 42;
SubscribeSensor subscribe_sensor = 50;
SetRosGraphEnabled set_ros_graph = 51;
SetOverlays set_overlays = 52;
}
}
message Play {}
message Pause {}
message Step { uint32 ticks = 1; int64 duration_ns = 2; } // one of the two; 0 = unused
message Seek { SimTime to = 1; } // replay only
message SetRealTimeFactor { double target = 1; } // 0 = unlimited
message Reset {}
message RequestSnapshot {}
message GetAttributes { repeated uint32 prims = 1; }
message SetAttribute { uint32 prim = 1; Attribute attribute = 2; bool save_to_session_layer = 3; }
message SetTransform { uint32 prim = 1; repeated double position = 2; repeated double quat_wxyz = 3; }
message SetVisibility { uint32 prim = 1; bool visible = 2; }
message LoadStage { string path = 1; }
message ImportRobot {
enum Format { FORMAT_UNSPECIFIED = 0; FORMAT_URDF = 1; FORMAT_XACRO = 2; FORMAT_SDF = 3; FORMAT_MJCF = 4; }
Format format = 1;
string path = 2; // path on the server's filesystem
DataRef content = 3; // or the file content, inline, when the viewer is remote
string overlay_path = 4; // on the server's filesystem
repeated string xacro_args = 5;
DataRef overlay_content = 6; // or the overlay's text, inline; added after 1.0
}
message OpenRecording { string path = 1; }
message RemoveRobot { string name = 1; } // added after 1.0 (RFC 0016 §7.2)
message CommandJoint { string joint = 1; string mode = 2; double value = 3; } // after 1.0 (§7.3)
// ImportRobot on lagrange_server (can_import): the robot is spawned into the running
// simulation, as simulation_interfaces' spawn_entity does (docs/guide/cli.md "Spawning
// robots"), at the origin of an empty run or 3 m beside the robots there are. By `path`, the
// result carries the path in imported_stage_path. By `content` (inline only; blob and shm are
// refused with "send the robot inline"), the text is written to a temporary file, the robot is
// named by its own <robot name> (SDFormat: the first <model name>; MJCF: <mujoco model>),
// renamed `name_2`, `name_3`… on a clash, and the result's imported_stage_path is its scene
// path, "/name": the name RemoveRobot takes. A `format` other than FORMAT_UNSPECIFIED must
// match the text. `overlay_path`, or `overlay_content` (inline YAML, or a SPICE netlist ending
// in `.end`), is applied to that robot as --overlay is (formats.md §9). `xacro_args` and
// FORMAT_XACRO are refused (run xacro first). The robot is passive (nothing drives it) when the
// run already has one; else it is the robot ROS drives. Both `path` and `content` is refused
// ("by path or content, not both"; older servers took the path and ignored the rest).
// Inline text is written, created exclusively and 0600, into $XDG_RUNTIME_DIR/lagrange (else
// <tmp>/lagrange-<uid>), a 0700 folder of the server's user; any other folder there is refused.
// RemoveRobot: as /delete_entity (cli.md "Spawning robots"), by top prim name with or without
// "/"; refused for the robot ros2_control drives and for scenery. A server older than these
// additions answers an unknown command with ok = false and an error, so a viewer can try it.
// CommandJoint: as /joint_commands under joint control (`lagrange_server --joint-control`),
// onto Server::commandJoint: `mode` "position" (rad, m), "velocity" (rad/s, m/s),
// "effort" (N·m, N) or "voltage" (V, a voltage-mode motor); the joint holds it until told
// otherwise. Refused, with the reason in `error`, for a joint the run does not have (fixed
// joints are merged at load), a mimic joint, a joint of a passive robot, an unknown mode, a
// value that is not finite, a mode the joint's motor cannot follow ("voltage" needs a motor in
// voltage mode), and a run without joint control or whose joints ros2_control or a vehicle or
// multicopter controller drives. A target belongs to its robot: removing the robot (or a
// reset) forgets it, and other robots coming and going keep it. The
// builder's Joints sliders send it while testing, at most 30 times a second per joint (RFC 0016
// §7.3); the mock server refuses it (a replay).
message ValidateProfile { string yaml = 1; }
message ApplyProfile { string yaml = 1; bool restart = 2; }
message GetProfile {}
message SubscribeSensor { string sensor = 1; bool enabled = 2; double max_rate_hz = 3; }
message SetRosGraphEnabled { bool enabled = 1; }
message SetOverlays { bool contacts = 1; bool forces = 2; string field_signal = 3; }
message CommandResult {
uint64 request_id = 1;
bool ok = 2;
string error = 3; // human-readable, one line
oneof result {
PrimAttributesList attributes = 10;
ProfileValidation validation = 11;
string imported_stage_path = 12;
}
}
message PrimAttributesList { repeated PrimAttributes items = 1; }
message ProfileValidation {
bool valid = 1;
repeated ProfileIssue issues = 2;
}
message ProfileIssue {
enum Severity { SEVERITY_UNSPECIFIED = 0; SEVERITY_ERROR = 1; SEVERITY_WARNING = 2; }
Severity severity = 1;
string json_pointer = 2; // "/domains/electrical/dt"
string message = 3;
}
4.6 profile.proto¶
syntax = "proto3";
package lagrange.viewer.v1;
// The profile itself travels as YAML text, the same text a user would save. The structured
// part below is what the domain panel needs in order to draw toggles without parsing YAML rules.
message ProfileState {
string yaml = 1; // the active profile, canonical formatting
uint32 schema_version = 2;
string json_schema = 3; // profiles/schema.json content, so the viewer validates with the server's schema
repeated DomainState domains = 4;
bool restart_required = 5; // the last ApplyProfile needs a restart to take effect
}
message DomainState {
string domain = 1; // "electrical"
bool enabled = 2;
bool available = 3; // false: cannot be enabled for this stage/tier
string unavailable_reason = 4; // "no ElectricalMotorAPI prims in stage"
string active_module = 5; // "averaged_network" or the substitute name
bool substituted = 6;
repeated SubstituteOption substitutes = 7;
repeated string backends = 8; // rigid_body: physx, mujoco, jolt (as installed)
repeated string fidelities = 9;
int64 dt_ns = 10;
int64 min_dt_ns = 11;
int64 max_dt_ns = 12;
double cpu_ms_per_sim_s = 13; // measured cost, for the panel's load bar
repeated string depends_on = 14; // domains whose signals this one reads
}
message SubstituteOption { string token = 1; string kind = 2; string description = 3; bool needs_source = 4; }
4.7 envelope.proto¶
syntax = "proto3";
package lagrange.viewer.v1;
import "lagrange/viewer/v1/common.proto";
import "lagrange/viewer/v1/session.proto";
import "lagrange/viewer/v1/scene.proto";
import "lagrange/viewer/v1/signals.proto";
import "lagrange/viewer/v1/command.proto";
import "lagrange/viewer/v1/profile.proto";
message Envelope {
oneof body {
ClientHello client_hello = 1;
ServerHello server_hello = 2;
ServerStatus status = 3;
Ping ping = 4;
Pong pong = 5;
LogRecord log = 6;
RosGraph ros_graph = 7;
SignalCatalog catalog = 10;
Subscribe subscribe = 11;
Unsubscribe unsubscribe = 12;
SignalBlock signal_block = 13;
SceneSnapshot snapshot = 20;
SceneDelta delta = 21;
Blob blob = 22;
SensorFrame sensor_frame = 23;
Command command = 30;
CommandResult command_result = 31;
ProfileState profile_state = 32;
}
}
4.1 Additions of 2026-10-14 (version 0.1, backwards compatible)¶
Added while building the viewer’s screens, as new messages and new fields only; buf breaking
passes against the previous files. The .proto files in proto/ are the reference; the
draft above predates them.
Addition |
Where |
Used by |
|---|---|---|
|
|
timeline event lanes (Scene, Replay), Events list |
|
|
Performance workspace |
|
|
Replay workspace: Recording and Channels |
|
|
Domains workspace: wiring graph, unit checks, schedule |
|
|
Electrical workspace: limits |
Rules that go with them:
An
EventListreplaces what the viewer holds; anEventappends. A recording sends its whole list withcomplete = true.ServerStats.estimatedsays whether module costs are measured or estimated. The mock server sets it: it replays, so its module costs are the profile’s estimates.ServerStats.Module.on_gpu(andWiring.Node.on_gpu) is true for a module whose step runs on a hardware GPU: a sensor casting on--lidar gpu’s ray context (lidar, radar), a sensor rendering on the cameras’ device when that device is not a software one (cameras, thermal cameras, event and stereo cameras, detections), and the aerodynamics with the live air box on the GPU. Plugin modules are never marked.cpu_ms_per_stepof such a module is the wall time of its step, waiting for the GPU included; the HAL has no timestamp queries yet, so there is no separate GPU time per module (that would need a new field, e.g.gpu_ms_per_step).ServerStats.gpu_memory_bytesis the memory the server’s hardware Vulkan devices hold through the HAL’s allocator (VMA’sblockBytesover every heap,gpu::Device::hardwareMemoryInUse): device-local images and buffers, and the host-visible buffers the GPU reads and writes (the GPU lidar’s rays live there). Exact for Lagrange’s allocations; the driver’s own overhead (pipelines, descriptor pools) and other processes are not in it. 0 when the server has only a software device (Tier 0) or none.Wiringis sent whenever the profile text could be parsed, also when a unit check fails: a wire withunit_ok = falseis what the user has to see (invariant 2).QuantityKindgainedLinearAcceleration, appended to the list (architecture §2).
4.2 Additions of 2026-10-14, second set (version 0.1, backwards compatible)¶
For the Environment, ROS 2 and World workspaces. Again new messages and new fields only.
Addition |
Where |
Used by |
|---|---|---|
|
|
Environment workspace |
|
|
field overlays and the probe |
|
|
World workspace |
|
|
the viewer’s camera as a hint for a streamed world’s tile selection, at most twice a second; ignored by deterministic runs |
|
|
ROS 2 workspace |
Rules that go with them:
FieldSlice,RosGraphandWorldStateare sent only to viewers that asked (SubscribeField,SetRosGraphEnabled,SetWorldEnabled); a viewer asks while a panel that shows them is open.World positions in
WorldStatearedoublein the world’s ENU frame (invariant 6). Tiles are parallel arrays and are sent again only whentiles_generationchanges.EnvironmentState.synthetic,WorldState.syntheticandRosGraph.notesay when a server describes something it does not model or run. The mock server sets all three: it has no environment model, no tile streamer and no ROS 2 bridge.An EMI source’s frequency is a parameter of the site or the device, never a default.
4.3 Addition of 2026-10-25 (version 0.1, backwards compatible)¶
Pre-approved by the owner on 2026-10-24 for autonomous work; listed for review in
docs/plan/progress.md.
Addition |
Where |
Used by |
|---|---|---|
|
|
viewport Collision shading |
Each collider’s mesh is its shape at its size, appended to
SceneSnapshot.meshesafter the visual meshes, soPrim.meshindices are unchanged. Planes are not sent.A collider moves with its prim,
0xFFFFFFFFfor fixed scenery. A viewer that does not know the field ignores it; the mock server sends none, and the menu entry is then disabled.
5. Session flow¶
sequenceDiagram
participant V as viewer (client worker thread)
participant S as server / mock server
V->>S: ClientHello {v1.0, wants_shm}
S->>V: ServerHello {v1.0, kind, capabilities, shm fds}
S->>V: ProfileState
S->>V: SignalCatalog {generation 1}
V->>S: Command RequestSnapshot
S->>V: SceneSnapshot {sequence 100} (+ Blob chunks or shm refs)
loop running
S->>V: SceneDelta {sequence 101.., base = previous}
S->>V: SignalBlock (subscribed signals)
S->>V: ServerStatus (10 Hz)
end
V->>S: Subscribe {signals for a new scope}
V->>S: Command ApplyProfile {yaml}
S->>V: CommandResult {ok}
S->>V: ProfileState, SignalCatalog {generation 2}
Note over V: subscriptions from generation 1 are void; the viewer re-subscribes by name
Clarifications fixed by the first implementation (the mock server):
Transforms in
SceneSnapshotandSceneDeltaare absolute in the region-local frame, not relative to the parent prim.A
SceneSnapshotalso marks a discontinuity in time. After a seek, a reset, or a looping replay restarting, the server sends a snapshot to every client, and signal sample indices restart from the new time after it. Between two snapshots, runs of a signal are contiguous.A
MODE_MINMAXrun has2 x componentsvalues per sample: all minima, then all maxima. A bucket is sent once, in the block in which its last sample falls.
Resynchronisation rules:
SceneDelta.base_sequencenot equal to the client’s sequence → the client drops deltas and sendsRequestSnapshot.A congested server skips deltas without breaking the sequence (
dropped_deltascounts them): its deltas carry the absolute pose of every moving prim and nothing else, so the next delta replaces a skipped one. A server whose deltas carry added or removed prims must instead break the sequence when it skips one.SignalBlock.catalog_generationolder than the client’s → the block is discarded.SignalRun.first_indexnot contiguous with the previous run → the scope draws a gap marker and the dropped-sample counter in the status bar increases. This is how the “no dropped samples” budget is measured.
7. Versioning and compatibility¶
Package
lagrange.viewer.v1. A breaking change createsv2in a new directory; both may be served during a deprecation window.Handshake: the server refuses a different
majorwithServerHello.refusalset to a sentence that names both versions. A differentminoris accepted; unknown fields are ignored (proto3 behaviour), and new messages are added as newoneofcases that old peers skip.Until v1.0 of the product the protocol major is
0and may break between minor releases. The viewer and server from the same release are always compatible. From v1.0 thev1package is frozen underbuf breaking.
buf breaking policy:
Phase |
Rule set |
Against |
Effect |
|---|---|---|---|
before v1.0 |
|
|
Breaking changes are allowed only in PRs labelled |
from v1.0 |
|
latest release tag |
No breaking changes in |
Lint: buf lint with the STANDARD rule set, except RPC_* rules (there are no services) and
with ENUM_ZERO_VALUE_SUFFIX set to _UNSPECIFIED. Nested enum values use a prefix of the enum
name, as in the drafts above; TODO(verify) the drafts against buf lint when the files are
created, since they were not linted in this planning task.
8. Conformance tests¶
tests/protocol/golden/: binary frames recorded from the mock server for each message. The C++ decoder and the Python encoder must both round-trip them.A Python reference client (
python/lagrange/viewer_client.py) connects to any server and checks the handshake, snapshot/delta sequencing, catalog generations and gap-free signal runs. The same script runs against the mock server in v0.1 and the real server in v0.2. This is what keeps the two servers interchangeable.Fuzzing: the C++ frame reader and
Envelopeparser get a libFuzzer target; malformed input must never crash the viewer (it receives data from the network).