flightopsDocumentation
Reference: recording

Independent exact-topic collector

5 min readdocs/reference/independent-collector.md (source repository)

The independent collector is separate from a managed application launch. It subscribes only to the 1โ€“16 exact topics in an enabled private recording policy for one canonical project/environment/ROS-domain scope. It neither starts nor controls the publisher application. A projectless scope is explicit null; it is not a fabricated project. The policy permits capture and retention, and denies sharing, inference, and training by default.

RecordingCatalog.configure_collector_scope(policy_path=..., policy=...) registers that scope stopped. install_service(config_path, unit_dir, ...) writes an owner-only configuration and a user-systemd unit, also inert. An explicit control_service(config_path, "start") sets the desired state to running and asks systemctl --user to enable/start the unit. pause closes the owned writer but keeps the controller and its scope lease; resume starts a new writer epoch. stop requests a graceful close, then disables/stops the unit. status reports desired state, persisted capture health, and a separate live systemd observation. If user systemd is unavailable, start restores the desired state to stopped and records systemd-unavailable; it does not claim supervision. run_config(config_path) is the foreground controller entry point used by the unit and does not by itself establish OS supervision. uninstall_service(config_path, unit_dir) requests stop, disables the exact unit, waits for the scope lease to clear, then removes only that private config and unit. It preserves catalog history and records uninstalled. If systemd is unavailable or the lease remains live, files remain and the returned state explains why the uninstall did not finish.

The controller holds a noninheritable scope flock and a SQLite generation/token lease. It never steals a lease from heartbeat age; an old owner must be proven dead or replaced by Linux process birth identity (boot ID plus start tick). Every collector lifecycle mutation checks its lease in the same transaction. If the supervisor died between spawning a writer and persisting that writer's PID, the arming reservation is ambiguous. The catalog refuses automatic takeover and uninstall cleanup rather than assume an orphan writer is gone; operator investigation is required. Known live writer groups also block dead-owner cleanup, including groups tied to a terminal session. The recording catalog refuses a managed-launch recorder in a scope reserved by a running or paused independent collector, including during rotation. A prior writer group that remains live prevents a replacement session.

Each finite batch uses OwnedRecorder's bounded rosbag2/MCAP writer. The catalog distinguishes desired paused from pause-requested; only the fenced controller acknowledges paused after its writer closes, and recording after the next writer arms. MCAP segments are finalized only after the whole writer group closes gracefully and the MCAP is read and hashed. A filename created by rosbag2 is still active until then. An epoch records its session link, discovery snapshot, unknowns, and terminal state. The rotation gap starts when one writer closes and remains open through ROS graph observation, capability probing, and the next writer's arming. A failed replacement leaves that gap unresolved. The bounded direct ros2 topic list --no-daemon snapshot is an observation, never proof of message coverage. Earliest publisher history and delivery during the gap are unknown. A finalized segment's message count is verified separately.

The standing policy's session, segment, total-byte, cache, free-reserve, shutdown-margin, and wall-duration limits remain enforced by the managed recording writer. A full budget or disabled/replaced policy stops capture visibly and exits cleanly, avoiding an automatic restart loop. An unexpected writer failure exits nonzero, subject to the unit's bounded restart rate (3 starts per 60 seconds, RestartSec=5). The unit uses KillMode=control-group and UMask=0077. Retention expiry and pins are available through an explicit scoped retention policy; without one the collector stops on a full budget. This controller captures selected topics from external nodes; it does not supervise or restart those nodes.

The owner-only config persists only known ROS/runtime environment keys. Unknown ROS_, RMW_, Fast DDS, or Cyclone DDS variables make installation fail, rather than silently widening discovery or security settings. The config, policy, lease, and user-service files must live outside project source. Backup may retain collector scope/epoch/gap history, but restore must clear lease ownership, set desired state to stopped, and mark service restored-inert; it must never restore a capture grant or activate a unit.

recording_schema version 2 migrates version 1 in place. It adds nullable collector scope/generation links to sessions and the collector_scopes, collector_epochs, and collector_gaps tables. History catalog PRAGMA user_version remains independent. A version 1 recording backup is accepted through migration. Unknown future recording schema versions refuse.

Public controls

After enabling an exact-topic standing policy, install and start the collector:

sh
flightops record enable --topics /robot/state
flightops record collector install
flightops record collector start
flightops record collector status
flightops record collector pause
flightops record collector resume
flightops record collector stop
flightops record collector uninstall

The project and ROS-domain flags precede collector. Installation prints the saved private configuration path. Pass --config /path/to/config.json after any other collector action to use it even when the selected ROS installation is unavailable. Start and stop refer to this collector, never the application.

Ordinary managed launches observe an installed collector's state and leave its lifecycle unchanged, including a deliberate pause or stop. Uninstall returns ownership to managed-launch recording under its current standing policy. Restored inert history also does not suppress explicit managed recording. A live lease or a running/paused desired state always reserves the scope.