flightopsDocumentation
Guides

Replay prerequisites and state-preserving slices

11 min readdocs/REPLAY-READINESS.md (source repository)

splashdown readiness inspects a saved recording against explicit replay prerequisites. It can also create a slice that retains static transforms and warm-up messages. Neither operation executes replay or establishes that an incident can be reproduced.

This is an offline splashdown feature. There is no readiness MCP tool and no automatic connection from this report to the replay engine's startup checks.

Declare the prerequisites

Use a separate JSON sidecar. The existing frozen scenario and replay-fixture formats are unchanged.

json
{
  "schema": "flightops.replay-prerequisites/0",
  "required_topics": ["/scan", "/odom"],
  "static_transforms": [
    {"parent": "base_link", "child": "laser"}
  ],
  "minimum_history_s": 3,
  "runtime_requirements": [
    "The controller lifecycle node must reach active before stimulus begins"
  ]
}

Only schema is required. Omitted lists default to empty and omitted minimum_history_s defaults to zero. A schema-only sidecar requests no topic, transform, history or resource requirements; its unblocked report is therefore a weak observation. Declare the prerequisites needed by the actual incident. Unknown fields and duplicate topic/runtime-requirement entries are refused.

required_topics checks recorded messages in the requested evaluation window. An absent channel reports messages: null; a recorded channel with no messages reports zero. static_transforms checks direct parent/child edges recorded by the start of that window. A transform first recorded later cannot establish startup state. Presence does not prove that geometry or calibration is correct.

minimum_history_s checks that the recording spans the requested warm-up window and each required topic has messages before evaluation starts. This establishes available history, not that the history is sufficient to initialize the algorithm.

Optional resources entries have role, path and sha256 fields. Use the actual 64-character lowercase SHA-256 of a map, parameter file, calibration file or saved startup state. Paths resolve under --root, which defaults to the sidecar's directory; references outside that root are refused. Finding the expected bytes does not establish that a node loaded them at runtime.

An optional clock object has recording, t0_ns and provenance fields. recording is wall or sim, and t0_ns is the run origin on that recording clock. This is a declaration with provenance, not a measured clock mapping. Without a declared origin or explicit --t0-ns, times are relative to the first recorded message. The report marks the missing clock mapping as unknown. runtime_requirements are preserved as unknown until supported by executed evidence. This command cannot establish lifecycle readiness or a successful reset.

Inspect or slice

bash
splashdown readiness preflight incident.mcap \
  --requirements replay-prerequisites.json --start 10 --end 20

splashdown readiness slice incident.mcap \
  --requirements replay-prerequisites.json --start 10 --end 20 \
  --topics /scan /odom /tf --output /tmp/incident-state-slice
ArgumentDefault and meaning
preflight or sliceRequired action. preflight inspects; slice also publishes a derived recording.
recordingRequired local regular MCAP path, at most 64 MiB.
--requirements PATHOptional JSON sidecar. Omission uses only the schema field.
--root PATHResource-file root. Defaults to the sidecar directory, or the recording directory if there is no sidecar. It does not confine the explicitly supplied recording path.
--start SECONDSDefaults to the first message time on the selected run clock. Explicit values must be finite.
--end SECONDSDefaults to the last message time on the selected run clock. Must be finite and at least the start.
--t0-ns INTEGERExplicit unsigned 64-bit recording-clock origin. If the sidecar also declares an origin, both must match.
--topics TOPIC ...Slice only. Defaults to all recorded topics. Does not replace required_topics in the sidecar. Unknown selected topics refuse slicing.
--output PATHRequired for slice. New directory with an existing parent; existing paths are never overwritten. Refused for preflight.

The evaluation interval is closed at both ends, [start, end], in seconds relative to source_t0_ns. Original MCAP log and publish timestamps are retained. This differs from splashdown query, whose absolute nanosecond interval is half-open. --start 10 --end 20 means 10 through 20 seconds after the chosen origin, not the absolute MCAP interval [10 ns, 20 ns).

Both commands emit JSON. Exit 0 means inspection completed and found no missing declared prerequisites; the report status remains unverified. Exit 3 means blocked, with missing observations, content mismatches or insufficient history. Exit 2 means the operation was refused or exceeded a resource limit. None is a successful replay verdict.

Runtime requirements and absent clock mapping remain unknown; they do not automatically make the offline report blocked. completeness: complete means the requested offline inspection completed, not that runtime prerequisites are known or that replay is reproducible. An invalid request or worker resource refusal prints flightops.replay-readiness/0 with status: refused and error. Argparse usage errors instead print usage to stderr.

The slice command publishes a new directory with recording.mcap and readiness.json. It retains:

  • Selected-topic messages in the evaluation window and the requested warm-up interval before it.
  • Every recorded message on /tf_static and namespaced static-transform topics through the requested end, even when those topics were omitted from --topics.
  • Original message payloads, schemas, publisher/channel metadata, sequence numbers, log timestamps and publish timestamps.

Keeping all prior static messages preserves transforms contributed by multiple publishers. It does not infer publisher completeness or merge their geometry. Messages beyond the requested end are excluded. A preamble alone cannot make an empty requested slice count as useful output.

The report records both the requested evaluation window and the expanded written window, plus the original source_t0_ns. It reopens the completed slice and checks its prerequisites again. Filtering out a required topic therefore produces a blocked slice report. The MCAP may still be useful evidence, so it is retained alongside that report.

Report fieldContents
schema, statusflightops.replay-readiness/0; unverified or blocked on completed inspection.
checksTopic, static-transform, history, resource, clock-mapping and runtime observations.
evaluation_window_s, source_t0_nsRequested closed interval and its original recording-clock origin.
sourceResolved source path, whole-file lowercase SHA-256, byte length and whether the recording reports an index.
requirements_sha256SHA-256 of the canonical parsed sidecar object, not its original whitespace.
static_topics, static_messages_inspectedDetected static topic names and the number inspected for required transform edges. No requested edges means the edge audit count can be zero.
completeness, reproducibility_verified, limitationsCompleted offline collection, explicit false reproducibility claim and remaining limitations.
sliceSlice-only file hash/size, message/topic counts, requested/written windows, dropped topics, warnings and prerequisite_retention.
slice_readinessSlice-only repeated prerequisite inspection of the completed derived recording.

Checks use available, missing, declared or unknown, with fields specific to the observation. Topic checks preserve channel_recorded and distinguish a missing channel's null count from an observed zero. Transform checks set geometry_verified: false; resources set loaded_at_runtime: unknown; clock checks set runtime_mapping_verified: false.

The slice's prerequisite_retention records original origin, requested evaluation window, requested/available history, static policy and retained timestamp policy. Its human-facing requested_window_s and written_window_s summaries round to milliseconds. The original message timestamps and prerequisite evaluation window remain the data to use for timing-sensitive follow-up.

Read the slice using the recorded origin when evaluating it:

python
import json
from pathlib import Path
from splashdown.recording import Recording

packet = Path("/tmp/incident-state-slice")
report = json.loads((packet / "readiness.json").read_text())
recording = Recording(packet / "recording.mcap", t0_ns=report["source_t0_ns"])

The report is not a replacement fixture. Creating a new replay fixture for the derived recording still requires its own reviewed timing, criteria and content hash. Do not attach the old recording's fixture hash to the new MCAP.

APIs and bounds

splashdown.readiness.preflight_recording(...) and splashdown.readiness.slice_with_state(...) expose the same bounded operations as the CLI. splashdown.slicing.slice_bag(...) also accepts opt-in preserve_static and history_s parameters. Its defaults remain unchanged. Direct slice_bag callers own their Recording construction and resource supervision; use the new readiness API for the child-process limits below.

python
import json
from pathlib import Path
from splashdown.readiness import (
    preflight_recording, slice_with_state, validate_requirements,
)

sidecar = Path("/evidence/replay-prerequisites.json")
requirements = json.loads(sidecar.read_text())
validate_requirements(requirements)
report = preflight_recording(
    "/evidence/incident.mcap", requirements=requirements, root=sidecar.parent,
    start_s=10, end_s=20,
)
slice_report = slice_with_state(
    "/evidence/incident.mcap", "/evidence/incident-state-slice",
    requirements=requirements, root=sidecar.parent,
    start_s=10, end_s=20, topics=["/scan", "/odom", "/tf"],
)

Both APIs return report dictionaries and raise ReadinessError or other input/OS errors on refusal. Python has no sidecar filename to infer a root from, so its default resource root is the recording's parent. Pass the sidecar parent explicitly when loading requirements elsewhere. slice_with_state always enables static retention and uses the sidecar's history request.

Direct slice_bag(recording, output, preserve_static=True, history_s=3) enables the same copy policy without a sidecar or a readiness report. Its preserve_static default remains false and history_s remains zero. Existing plain slices therefore keep their earlier behavior. Direct callers must still review the output and create a fixture that describes the new recording.

The readiness APIs perform MCAP work in a POSIX child with a 30-second wall-clock deadline, 20-second CPU allowance, 768 MiB address-space limit and 64 MiB output-file limit. Inputs are capped at 64 MiB, 250,000 messages and 256 topics. Decoding a malformed compressed chunk can terminate this child but cannot exhaust the caller's address space. Larger bags require the separate bounded evidence-query path or a reviewed smaller recording; this command does not raise all existing limits.

Prerequisite JSON is at most 64 KiB. Lists allow at most 64 items, except 128 static edges. Requested history is at most 300 seconds. Resource files are individually bounded at 32 MiB and together at 64 MiB. Reports are limited to 512 KiB. Unknown schema versions, duplicate JSON keys, non-finite numbers, terminal controls, nonregular files and partial recordings are refused. Out-of-root resource paths produce missing-resource checks without reading the referenced file.

Slice publication stages both files, reserves a new destination exclusively, then publishes the directory atomically. Existing files are not overwritten. Failed or interrupted worker operations leave no published packet.

The existing replay engine still uses timed recorder settling and entrypoint liveness checks. This change adds no runtime-startup gate and makes no ROS/Gazebo capability claim. A real failure reproduced on another clean machine remains a separate acceptance test.

Troubleshooting and verification

SymptomInterpretation and next step
topic check is missing with messages: nullThe channel was not recorded. Collect that topic in new evidence.
topic check is missing with messages: 0The channel exists but has no messages in the requested interval. Check the chosen origin/window and actual delivery.
Static edge missingThe requested direct parent/child edge was not observed by evaluation start. A later transform or an indirect chain does not satisfy this check.
History missingThe recording begins too late, ends before evaluation starts, or a required topic lacks a warm-up message. Extending a slice cannot manufacture missing history.
Resource content mismatchCompare the declared SHA-256 with the exact intended map/configuration/calibration bytes. Do not infer runtime loading from a file match.
Resource escapes rootMove the resource under the declared root or choose the correct root explicitly. The external path is not read.
Explicit origin differs from sidecarUse one reviewed clock declaration. The command refuses conflicting origins.
Slice refused as emptyThe selected topics have no messages in the evaluation window. Static preamble or warm-up messages alone cannot rescue the request.
Slice report becomes blockedInspect slice_readiness; filtering may have removed a required topic that the source recording contained. The blocked packet remains available.
Worker failed or exceeded its budgetUse a smaller supported recording. The APIs require POSIX process/resource support; they do not provide an equivalent Windows worker.

The resource limits protect the child process, including the existing whole-recording timestamp index. Readiness does not use the large-bag indexed query implementation. A schema-only requirement file still causes recording inspection and must fit the same bounds.

readiness.json is not currently an input to flightops-results or flightops-tui. Inspect its JSON directly. Those viewers accept canonical gate proofs, comparison records and evidence-window observations, not every Flightops schema. A derived MCAP can be queried or checked through the normal recording tools, with its retained origin supplied where required.

The regression tests need a source checkout: they are in splashdown/tests/, which is not in the install. From the repository root:

bash
.venv/bin/python -m pytest \
  splashdown/tests/test_replay_readiness.py splashdown/tests/test_slicing.py -q

The regressions exercise multiple static publishers, transforms arriving after evaluation starts, missing history, missing versus zero-count topics, resource hash mismatches, preserved timestamps, slice reinspection, malformed requests, compressed input bounds and failed publication. They do not execute a controller, establish lifecycle readiness or reproduce a field incident.