PAANI: On-Device Visual Evidence Fusion for Explainable River Robot Navigation

The Core Challenge for River Monitoring Robots

Mobile river monitoring robots operating in real waterways cannot rely on GPS waypoints alone to handle complex environments — they must recognize obstacles and identify water surface boundaries in real time, information that geographic coordinates simply cannot capture. Making things harder, these robots typically run on compute-constrained edge platforms. Translating imperfect visual predictions into timely, verifiable navigation commands is an independent and serious engineering challenge in its own right.

The PAANI architecture proposed in this paper addresses a long-neglected pain point: an object label or steering command on its own cannot explain what evidence supports the decision, nor can it tell an operator under what conditions that evidence becomes unreliable. In other words, conventional approaches deliver black-box conclusions, whereas aquatic operations demand traceable, explainable reasoning.

PAANI's Technical Architecture

PAANI is an end-to-end "perception to navigation" on-device architecture deployed on resource-constrained hardware such as the Arduino UNO Q. It combines two lightweight vision models: a custom-trained YOLO11n detector for object detection and a custom MobileNetV3 Small semantic segmenter for water surface delineation. The outputs of both models are integrated through a timestamp-aligned evidence fusion mechanism.

Built on top of this, the system introduces bounded tracking to maintain persistent object identity and prevent decision jitter caused by single-frame missed detections. What truly embodies the "explainable" philosophy is the explicit corridor policy: this policy synthesizes water surface labels, accepted detections, urgency levels, and segmentation mask uncertainty to produce navigation recommendations.

Every Recommendation "Shows Its Evidence"

PAANI's most distinctive design choice is that every final navigation recommendation exposes the supporting evidence and the policy reasoning behind it. This means operators see not just a command like "turn left to avoid obstacle," but also why the system made that decision and which visual evidence it drew upon. This inspectability is especially critical for safety-sensitive aquatic robots.

The system also connects the local AI pipeline to a standalone Gazebo simulation vessel, localization, and control testbed via a ROS 2 interface, forming a complete simulation validation loop.

---

YOLO11n is the latest nano-scale variant in the Ultralytics YOLO family — extremely small in parameter count and designed specifically for edge inference. The YOLO (You Only Look Once) architecture simultaneously predicts bounding boxes and class probabilities in a single forward pass, eliminating the region proposal step required by two-stage detectors like Faster R-CNN, which gives it a significant speed advantage on low-compute devices. MobileNetV3 Small is Google's mobile-optimized lightweight backbone, leveraging hardware-aware neural architecture search (NAS) and a redesigned activation function (h-swish) to balance accuracy and computational cost. Combining the two — the former for object localization and classification, the latter driving pixel-wise semantic segmentation — is a common lightweight strategy for obtaining both "what is there" and "where is it" simultaneously on single-board-class hardware.

ROS 2 (Robot Operating System 2) is the dominant middleware framework for modern robot software development, providing inter-node publish/subscribe messaging, service calls, and action interfaces. Compared to the original ROS, ROS 2 is built on DDS (Data Distribution Service) with real-time guarantees and multi-robot coordination support. Gazebo is a 3D physics simulation platform deeply integrated with the ROS ecosystem, capable of simulating hull dynamics, water surface environments, and sensor noise — allowing researchers to validate control and perception algorithms without a physical vessel. PAANI publishes the output of its onboard AI pipeline as standard ROS 2 topics; the virtual vessel in Gazebo subscribes to these topics and executes the corresponding actions, forming a complete "perception → decision → simulated execution → feedback" validation loop that reduces the cost and risk of real-world water testing.

Training Data and Model Performance

On the data side, the detection task was trained on 10,000 WaterScenes images across four object categories; the segmentation task used 1,127 MaSTr1325 images, including 198 segmentation validation images. The final FP32 ONNX model occupies just 14.817 MB, fully reflecting the lightweight requirements of edge deployment.

In terms of performance, the detection checkpoint achieves a test mAP of 0.7388 at IoU 0.5, with the separately evaluated rectangular ONNX export achieving a validation mAP of 0.7367 — highly consistent with each other. The segmentation ONNX achieves a validation mIoU of 0.9750, a strong result.

Real-World Latency on Edge Hardware

A five-minute recording captured on the UNO Q — configured at a 0.5 Hz sampling rate — showed a pipeline latency median of 467.8 ms and a 95th percentile of 580.3 ms. This honestly reflects the execution cost of running multi-model fusion on low-cost hardware. While not fast, it is acceptable for a 0.5 Hz monitoring scenario.

---

mAP (mean Average Precision) is the standard evaluation metric in object detection, computed as the mean area under the precision-recall curve for each class at a specified IoU threshold. An IoU of 0.5 means the predicted bounding box must overlap the ground truth by more than 50% to count as a correct detection. mIoU (mean Intersection over Union) is the corresponding metric for semantic segmentation, measuring the average ratio of intersection to union between predicted and ground-truth masks across all semantic classes — the closer to 1, the more precise the pixel-level classification. An mIoU of 0.9750 is relatively high for a small dataset, though it should be noted that the MaSTr1325 dataset is primarily composed of European coastal and inland waterway scenes, and visual domain gaps with specific deployment rivers may affect generalization. ONNX (Open Neural Network Exchange) is a universal intermediate representation for cross-framework model deployment; FP32 refers to storing weights as 32-bit floating-point numbers, preserving full precision — suitable for edge devices that do not yet support hardware quantization.

Honest Disclosure of Limitations and Failure Cases

One commendable aspect of this work is that it does not sidestep the system's shortcomings. The evaluation identified two explicit failure modes: black input misclassification — where pure black or extremely dark frames trigger incorrect predictions — and sampling rate mismatch, which prevents the apparent motion estimator used for diagnostics from collecting sufficient evidence.

The authors specifically emphasize that these results support an "inspectable, reusable edge robotics baseline platform," and explicitly distinguish between three separate claims: model accuracy, onboard execution capability, and collision avoidance capability validated in real water. In other words, PAANI currently validates the first two; real-world aquatic collision avoidance still requires further verification. This kind of clearly bounded claim is rare and commendable in robotics research.

Implications for Edge Robotics Research

PAANI's value lies not just in its specific performance numbers, but in the methodology it offers: how to transform imperfect perception results into explainable, traceable navigation decisions on compute-constrained edge platforms. It draws a clear line between "model capability" and "real-world deployment effectiveness," avoiding the common trap of overclaiming.

For researchers working on aquatic robotics, edge AI deployment, and explainable AI, this architecture — combining YOLO11n, MobileNetV3, evidence fusion, and ROS 2 simulation — provides a reproducible reference baseline. It demonstrates that lightweight models (under 15 MB) can run a usable perception-navigation pipeline on real hardware while maintaining transparency in the decision-making process.