Solve ROS 2 problems with Flightops: illustrated examples

Use these six examples to move from a symptom to evidence, a proposed fix, and a repeatable check. They come from Nav2, Panda/MoveIt, and Go2 simulation packages in a separate test lab with injected faults.

The lab is not shipped with Flightops, so you cannot rerun these exact cases. Each example is a walk-through of the evidence captured from the lab: the command that was run, the screenshots, and how to read them. Use the same steps on your own robot.

What actually happened: Flightops helped diagnose the injected faults using topic connections, bounded message samples, and original launch logs. The faults were not repaired during these reviews. Each “Fix to try” below is a next step, and each “Verify” describes evidence still needed after that fix. Healthy runs are separate baselines, not proof that a faulty run was repaired.

Open the HTML guide for page search and section navigation. Screenshot files are unmodified; their original paths, dimensions, build scope, and SHA-256 hashes are in the provenance record.

How the evidence was captured

The lab was a built ROS 2 Jazzy/Gazebo workspace with three packages: flytops_nav_lab, flytops_arm_lab and flytops_dog_lab. Its setup helper gave each lab its own ROS domain and Gazebo partition. Each case was run from the lab directory, with the lab overlay sourced, one scenario at a time:

flightops --project . --new-session launch PACKAGE LAUNCH_FILE

The healthy launches contain automatic motion probes; their PASS output was retained before a fault was tried. Pressing q closes Flightops and stops its owned launch. Reopening retained results does not start the launch again.

The screenshots marked R001 and R002 come from an earlier installed build. The screenshots marked R003 come from an earlier development build of the terminal. The controls below are the ones in the terminal today.

Discovery counts describe a timestamped capture whose completeness may be unknown. A discovered node does not prove its controller is active. A failed sample does not prove a sensor is broken. First-error and child summaries are pattern signals; read their original context before choosing a fix.

Nav2: map server cannot load its YAML

Package / symptom: flytops_nav_lab, missing_map.launch.py. The map server reports a load/configuration failure, so navigation cannot use its map.

Recorded run (nav lab): flightops --project . --new-session launch flytops_nav_lab missing_map.launch.py

Use Flightops: Open L, search warehouse_map, press c for surrounding lines, and pan right to read the complete path. Match the filename with the load failure and lifecycle configuration output. Read the raw retained path shown by the log view if more detail is needed.

Screenshot evidence — R003 earlier development build: The original output names warehouse_map.yaml and says Failed to load map yaml file. The view is horizontally panned, so the left part of the header is outside this frame. Open the full-resolution screenshot.

How Flightops helped: It recovered the early map failure from retained history instead of relying on recent launch output. This run also had a separate Gazebo child exit 255; do not attribute that exit to the map fault.

Fix to try: Correct the map YAML reference in the launch/configuration, ensure the file and its referenced map image are installed and readable, then rebuild/source the lab overlay if needed.

Verify: Repeat the affected launch with the corrected configuration. Retain the absence of the map-load failure, map server activation, a usable map observation, and a navigation probe result. Resolve the separate Gazebo exit before claiming a complete simulation pass.

Nav2: AMCL listens to a different lidar topic

Package / symptom: flytops_nav_lab, lidar_topic.launch.py. Localization has a scan connection mismatch to investigate.

Recorded run (nav lab): flightops --project . --new-session launch flytops_nav_lab lidar_topic.launch.py

Use Flightops: In 3 Topics, search lidar, select /lidar/scan, and read Connections. Clear the filter, select /scan, and press i for bounded messages. Compare exact names, message type, endpoint direction, graph times, and sample times. Follow the subscription to /amcl through Connections.

Screenshot evidence — R001 installed baseline: /amcl subscribes to /lidar/scan; this capture reports zero publishers and one subscriber. Open full resolution.

Screenshot evidence — R001 installed baseline: A separate bounded inspection collected LaserScan samples on /scan. Open full resolution.

How Flightops helped: Comparing the subscribed name with an observed message stream exposed a concrete routing mismatch. These captures do not establish the /scan publisher's identity, prove AMCL received these messages, or rule out QoS/TF issues. The later R003 lidar run had Gazebo exit 255 and unavailable samples; it does not replace the earlier successful sampling evidence.

Fix to try: Align AMCL's scan topic/remapping with the intended laser producer. Check the producer identity and QoS before changing configuration.

Verify: Capture both endpoints on the same intended topic, collect new samples with times/frame IDs, check required transforms, and run a localization and navigation probe. Endpoint presence alone is not proof of delivery.

Panda: MoveIt requests an unavailable planner

Package / symptom: flytops_arm_lab, bad_planner.launch.py. MoveIt cannot load its requested planning plugin and terminates.

Recorded run (arm lab): flightops --project . --new-session launch flytops_arm_lab bad_planner.launch.py

Use Flightops: Open L, search planner, enable c, and pan to read the complete plugin name. Read the exception and declared available plugin types. If the first-error summary points to an octomap message, continue to the planner failure; the summary does not determine the root cause.

Screenshot evidence — R001 installed baseline: The exception names flytops_nonexistent/PlannerManager and lists declared planner classes. The baseline log view displays producer terminal escape text; the current terminal display cleans these controls. Open full resolution.

Screenshot evidence — R003 earlier development build: Searching planner finds the original fatal lines 32/34 after 41,483 bytes of output. Long text needs horizontal pan; open this full browser capture at its original resolution. Open full resolution.

How Flightops helped: It connected the termination to the named missing plugin and kept the early failure reachable after later Gazebo output.

Fix to try: Choose an installed planner class advertised by pluginlib (for example ompl_interface/OMPLPlanner if available), or install/build the intended plugin and source its overlay. Check the selected planning pipeline.

Verify: Repeat the corrected affected launch; retain plugin initialization, a successful plan and execution, and actual joint movement. The separate healthy Panda R003 run observed a PASS with 0.365 rad change; it is a useful baseline, not a post-fix result.

Separate healthy baseline — R003 earlier development build: Successful motion is stronger evidence than merely discovering /move_group. Open full resolution.

Panda: controller requests a missing joint interface

Package / symptom: flytops_arm_lab, wrong_joint.launch.py. A controller node is discoverable even though activation failed.

Recorded run (arm lab): flightops --project . --new-session launch flytops_arm_lab wrong_joint.launch.py

Use Flightops: Look at 4 Nodes, then return to 1 Health to check child failures. In L, search not available or panda_joint8; inspect the interface warning and spawner failure together. Use all severities (a): the original unavailable-interface line is labelled WARN by the producer.

Screenshot evidence — R002 installed baseline: Discovery alone did not establish successful controller activation. Open full resolution.

Screenshot evidence — R002 terminal recall: The retained original log says panda_joint8/position is unavailable and the spawner exited with code 1. This frame is terminal recall of the retained log, not the L log view. Open full resolution.

How Flightops helped: The discovered node and its retained failure output could be investigated separately. The terminal also surfaces failed children independently of an observing parent and unknown readiness.

Fix to try: Align controller joint names and command interfaces with the robot's actual ros2_control hardware description. Rebuild/source changed installed configuration as needed.

Verify: Confirm successful controller activation, the expected action endpoint, a successful execution request, and joint-state movement. Retain the new log and samples; a node row or a successful plan alone is insufficient.

Go2: a required position interface is unavailable

Package / symptom: flytops_dog_lab, missing_position_interface.launch.py. The leg controller cannot activate; the automatic walk probe fails before walking.

Recorded run (dog lab): flightops --project . --new-session launch flytops_dog_lab missing_position_interface.launch.py

Use Flightops: Read child failures in Health, then search position and FAIL in L. Read the controller activation error, spawner exit, and probe result in their original context. Keep parent state and declared readiness separate from these failures.

Screenshot evidence — R001 installed baseline: The original output identifies unavailable lf_hip_joint/position, failed activation, spawner exit 1, and FAIL: Leg trajectory controller did not become active. Open full resolution.

How Flightops helped: It narrowed the problem to activation of a named joint command interface, before gait tuning or walk-distance investigation. R003 terminal text also confirmed child/probe failures separately from parent state; no JPEG was captured for that later run.

Fix to try: Restore the expected position command interface for the named joint in the robot's ros2_control description, or align the controller with the intended hardware interface contract.

Verify: Retain successful activation and a new walk probe with movement and joint cycling. The separate R003 healthy Go2 run passed with 3.112 m travel; it does not prove this missing-interface fault was repaired.

Go2: commands go to the wrong trajectory topic

Package / symptom: flytops_dog_lab, wrong_control_topic.launch.py. Trajectory messages are produced, but the intended controller is subscribed to a differently named topic and the walk probe fails.

Recorded run (dog lab): flightops --project . --new-session launch flytops_dog_lab wrong_control_topic.launch.py

Use Flightops: In 3 Topics, search trajectory. Read the full names and Connections for both /joint_group_position_controller/joint_trajectory_typo and /joint_group_position_controller/joint_trajectory. Select the producer topic and press i to inspect actual trajectory values. Read the walk probe in L rather than inferring movement from publisher counts.

Screenshot evidence — R002 installed baseline: The _typo topic has a captured producer connection from /quadruped_controller_node, one publisher, and zero subscribers. Open full resolution.

Screenshot evidence — R002 installed baseline: The receiving controller uses the topic without _typo. These endpoint captures were taken at different times and do not establish a complete simultaneous graph. Open full resolution.

Screenshot evidence — R002 installed baseline: Bounded trajectory samples show the producer emitted commands. This is not proof that the controller received or executed them. R003 inspected action/status unsuccessfully; its run does not claim new successful trajectory samples. Open full resolution.

Screenshot evidence — R002 installed baseline: The walk probe reports FAIL and only 0.215 m of ground-truth travel: the probe failed its acceptance threshold despite measurable displacement. R003 retained the same travel-failure value in terminal text. Open full resolution.

How Flightops helped: It made the producer name, consumer name, sample values, and behavioral failure available in one investigation. The terminal's suffix-preserving rows make the long names easier to distinguish.

Fix to try: Route the gait producer's trajectory output to the intended controller subscription, after checking message type and QoS compatibility.

Verify: Capture current matching endpoints, new bounded samples at the intended topic, controller activation, actual joint movement, and a passing walk probe. Keep those new artifacts separate from this faulty run.

Retain the diagnosis and check the repair

For every case, save the exact command and configuration/build identity, graph observation time, each inspection attempt's collection time/outcome, original log path/hash, and the behavioral probe output. Use 9 Capture to read export success/failure, the original selected object, and the full destination. Check the exported artifact itself.

For a log whose important lines need a text receipt, use the evidence CLI:

flightops evidence --receipts /absolute/path/to/receipts register /absolute/path/to/launch.log
# Copy receipt_id from the registration result.
flightops evidence --receipts /absolute/path/to/receipts read RECEIPT_ID --kind lines --start 31 --end 35

The example range is lines 31–34: start is inclusive, end is exclusive. Line recall prints readable text before its digest/range metadata; add --json to read for machine-readable exact bytes. Do not assume the same line numbers in another run. L searches retained evidence only; bounded retention can omit middle history and marks gaps.

These examples reach diagnosis and a proposed repair; none shows a repaired run. For the terminal controls, see the user guide. For the exact log retention limits, see launch log summaries.