flightopsDocumentation
Guides

Bounded session TF history

4 min readdocs/TF-HISTORY.md (source repository)

ManagedTFHistory keeps one ROS tf2 buffer and TransformListener running while an observation session collects messages. A later selection can query the exact saved header timestamp even after the dynamic publisher stops. This extends the existing fresh-listener message-TF lookup; its behavior and API remain available.

In the Rust workspace, start collection before sampling the topic:

sh
flightops-tui live --project /work/robot --ros-container my-ros \
  --ros-domain-id 42 --target-frame base_link --tf-history-seconds 30 \
  --session-dir /work/robot/.launchcommit/camera-session

Press i on the selected topic, then t to query that message against the session buffer. Add --tf-use-sim-time for simulation time. The workspace worker lifetime defaults to one hour; --tf-history-lifetime can reduce it. Reopen the session after expiry or budget exhaustion. Without --tf-history-seconds, the original fresh listener behavior remains. --json only prints configuration and starts no worker.

python
from flightops_tools.terminal_runtime import RosAdapter
from flightops_tools.tf_history import ManagedTFHistory

ros = RosAdapter(container=None, domain_id=42)
with ManagedTFHistory(ros, cache_seconds=30, lifetime=300,
                      max_transforms=100000, use_sim_time=False) as history:
    # Collect the topic inspection AFTER the listener starts.
    inspection = collect_selected_topic()
    result = history.lookup(inspection, 'map', message_index=0, timeout=3)

The caller owns collection and session lifetime. start() starts a worker and returns the session; close() is idempotent and closes its input, waits for normal ROS cleanup, then terminates the local process if necessary. Context-manager exit also closes on exceptions. A closed session cannot restart; create another object. Docker uses interactive docker exec stdin, so owner EOF reaches the same worker. Abnormal host/network failure may delay remote cleanup until the bounded lifetime.

The worker reports ready when subscriptions exist; DDS discovery or receipt is not yet proven. Start it before the relevant TF and message window. The query waits up to its monotonic deadline and returns a specific missing-frame, disconnection or extrapolation outcome when needed. No ROS-time sleep can stall on a paused clock.

For an independent collection window:

bash
flightops-tui tf-history /tmp/topic-inspection.json \
  --target-frame map --ros-domain-id 42 --collect-seconds 10 --cache-seconds 30

Save the inspection during that 10-second window; the command reads it afterwards. --ros-container NAME runs the self-contained worker inside a ROS container. --use-sim-time declares the session clock mode. JSON goes to stdout; startup goes to stderr. Exit 0 means a transform was available, 1 unavailable, 2 invalid input. This command does not launch the robot, publish TF, replay a bag or collect messages.

Bounds and evidence

  • Cache: 1–300 seconds of tf2 dynamic history; default 30 seconds.
  • Worker lifetime: 1–3600 monotonic seconds; default 300.
  • Accepted dynamic and static transform budget: 1–1,000,000; default 100,000. On exceeding the lifetime count, the buffer clears and every lookup returns history_limit. This is a conservative total-ingestion budget, not a claim of exact memory usage or a rolling message-count cache. Restart to collect anew.
  • Protocol: bounded JSON lines, 64 KiB request/response; one outstanding query per session. CLI inspection files are capped at 1 MiB.
  • Results use flightops.message-tf/0, including source/target, requested stamp, environment, declared clock, session identity, configured limits and collection start time. Existing results presentation accepts them.

Zero timestamps are rejected because tf2 interprets them as “latest.” There is no latest fallback and no custom transform math. The observed message's container and ROS domain must match when recorded. Recorded clock.use_sim_time, when present, must match. Missing clock provenance remains caller-declared; equal clock mode and domain do not prove the same recording or clock epoch. tf2 handles backward clock jumps with its buffer-clearing policy. History is in memory only, cannot be resumed across sessions and does not recover dynamic transforms preceding listener startup. A successful lookup proves availability at the requested time, not calibration, message delivery, safety or correct robot behavior.

Verification

Offline tests exercise the real subprocess protocol with a fake ROS endpoint, repeated requests to the same worker, exact stamps, environment/clock mismatch, zero-stamp refusal, bounds, worker bootstrap and exception cleanup. They do not prove real tf2 behavior.

Opt-in live test, in a sourced ROS environment. It needs a source checkout: it uses integrations/tests/, which is not in the install. Run it from the repository root:

bash
PYTHONPATH=integrations:onscope:splashdown:launchcommit:highbay:axikit \
python3 integrations/tests/tf_history_live.py --ros-domain-id 173 > tf-history-live.json

Use an isolated domain. The helper publishes only uniquely named test TF frames; it commands no hardware. It verifies historical lookup after publishing stops, a fresh-listener negative control, old-stamp extrapolation after cache eviction, transform-budget exhaustion and normal worker cleanup. Runtime compatibility is unproven until this helper passes in the selected ROS environment.