flightopsDocumentation
Reference: recording

Managed recording

6 min readdocs/reference/managed-recording.md (source repository)

The managed recorder captures an exact, selected ROS topic set during an owned launch. Enable a standing scope once with save_enabled; subsequent eligible managed launches arm capture before the application starts, without pressing Save or using a model. disable revokes the standing scope. This recorder stops with its managed launch. To capture topics from external nodes, with a supervised restart, use the independent collector. For retention budgets, expiry and incident pins, see recording retention.

Public Python API

recorder_policy.load_policy(path) returns a disabled policy when no policy file exists. save_enabled(path, *, project_root, environment_identity, ros_domain_id, topics, shared_topics=(), bounds=None) writes a private, digest-bound policy. project_root=None is a valid environment-only scope. disable(path) writes a disabled policy and clears its active scope. The policy file must be owned by the current user, mode 0600, outside project source, and free of parent symlinks. Topics must be exact absolute names; /rosout, /tf, and /tf_static need separate whole-stream inclusion in shared_topics. The fixed source-use decisions permit capture and retention only; sharing, inference, and training are denied.

OwnedRecorder(catalog_path, policy, environment, directory, *, subprocess_env, policy_path=None, ros2_executable='ros2') accepts a global recording artifact root as directory and creates one unique session directory under it. environment is scope identity with project_root, environment_identity, and ros_domain_id. Required subprocess_env is the same sanitized ROS/RMW/domain environment used by the application launch; the recorder never substitutes its ambient process environment. Supply policy_path so the monitor stops capture on disable or replacement.

Call start() before starting the application, then application_started() after the application reports successful spawn. Call close() after the application closes. Each returns a bounded state dictionary with state (writer_state is the same lifecycle), capture_status, reason, session ID, arming/application timestamps, segment summaries, unknowns, gaps, and log tail. capture_status is armed_payload_unverified before the first size observation, then writes_observed_payload_unverified after MCAP bytes grow. Byte growth can be headers and does not prove any message payload. The snapshot also gives observed_mcap_bytes, last_size_observed_at, and last_growth_at. Only closed, read-verified segments yield closed_verified_payload or closed_no_payload; interrupted closure is closed_unverified. Failure of capture never starts, stops, or restarts the application. close() is idempotent.

probe_recording_capability(*, subprocess_env, ros2_executable='ros2') checks the installed ros2 bag record flags and runs an actual rosbag2_py.SequentialWriter with MCAP storage in a temporary directory. It writes a synthetic payload directly to storage, with no ROS graph subscription or publication, then reads the closed MCAP. Installed package presence or --help alone is not a usable-writer result. The probe and real writer use the supplied subprocess environment.

RecordingCatalog(path) uses the same SQLite file as HistoryCatalog, with an independent recording_schema metadata table. It exposes create_session, mark_session, create_active_segment, finalize_segment, get_session, list_sessions, list_segments, artifact_inventory, retained_bytes, and reconcile_dead_owners. SQLite is the sole authority for recording lifecycle. MCAP files are byte artifacts, not a second lifecycle manifest. A dead writer is never resumed; its active segments become interrupted only when the stored process birth identity is proven dead or replaced. An unprovable owner remains explicit.

Bounds and evidence meaning

The enabled policy has separate limits for total retained bytes, session bytes, segment bytes, minimum free disk, writer cache, log bytes, shutdown margin, and wall duration. Rosbag2's cache can be double buffered, so the shutdown margin must cover at least twice the configured cache and be at least 2 MiB. Smaller configured values are refused before a writer is armed. The 2 MiB floor is based on a bounded native 8 MiB tmpfs trial where a 2 KiB margin left an interrupted, unreadable MCAP and a 2 MiB margin allowed a verified close. It is a minimum configuration guard, not a guarantee of closure under every write rate or filesystem. The writer cache must also be at least 1 MiB: in isolated native ext4 runs at 250โ€“500 Hz with 16 KiB payloads, a 1 KiB cache dropped messages that rosbag2 counted as lost, while 1 MiB and the default 4 MiB caches showed no gaps. A smaller cache in an enabled policy, including a previously saved one, is refused until the policy is reconfigured. The monitor samples every 250 ms; bytes written between samples and during graceful shutdown can exceed the trigger. Measure this overshoot on the target machine and set a machine-specific resource threshold before you treat a bound as a hard ceiling. Both the per-session and total budgets count all files in registered session directories, including metadata and unindexed interrupted files. Exhaustion stops capture visibly and does not delete older evidence.

Active segments have no final SHA-256. Only a gracefully stopped writer group whose MCAP files read through a valid summary and message iterator may produce finalized segment digests. A crash, forced stop, truncated file, or failed verification remains interrupted/unverified. A valid zero-message MCAP is retained with an explicit no-payload unknown. armed_at and app_started_at are wall-clock times; first_recorded_log_ns is the bag message log clock and must not be compared to wall time without a verified clock relationship. Dynamic discovery may miss initial messages even when arming preceded application start. No full-coverage claim follows from armed_at <= app_started_at alone.

Each segment stores its historical lifecycle state, original path, digest, and separate availability. availability='catalogued' means the path has not been rechecked for a later read. Backup restore can mark a finalized segment restored, excluded, missing, or changed without changing its historical result. resolved_path is null for excluded/missing/unverified bytes. Recording segment bytes must appear explicitly in backup inventory; a database-only copy is not a complete recording backup. Historical session policy is not current permission to export or copy a recording.

Log and clock capture boundaries

Every session now projects flightops.capture-boundaries/1 from its saved policy, segment finalization and topic index. The recorder status, desktop workspace and selected investigation expose the same projection. It adds no separate storage authority and does not read or reverify payload bytes.

/rosout selection, whole-stream authorization and messages verified at segment finalization are separate facts. Missing topic indexes remain unknown. A zero count applies only to fully indexed finalized segments; it does not establish continuous capture, logging configuration, or absence of suppressed log messages. Managed application stdout/stderr remains explicitly unlinked when the session has no log receipt. External process stdout/stderr is outside recorder ownership. The bounded live log_tail is labelled recorder stdout/stderr and is not a retained application log.

Lifecycle UTC, MCAP log-time nanoseconds, sensor/header time and ROS /clock remain separate clock domains. The projection records every pair as unknown without a calibration receipt. Recording /clock does not establish node use_sim_time, timestamp decoding or clock alignment. Historical finalization facts survive payload expiry; current byte availability must be checked separately.

Projection reads use one SQLite snapshot and inspect at most 1,001 segment rows. Sessions over the 1,000-segment display budget return unknown counts and ranges. Partially indexed sessions label verified counts as partial lower bounds.