BiRefNet Mobile NPU Deployment Challenges: Why Qualcomm Chips Struggle with Sub-Second Segmentation

Real-time high-quality image segmentation on mobile devices has always been a challenge for on-device AI deployment. Recently, a developer posted on Reddit asking for help, saying they tried to deploy the BiRefNet model to Qualcomm mobile GPUs and NPUs, but compilation to the NPU failed entirely, and performance on the GPU was also unsatisfactory. Their project goal is to achieve sub-second latency on these chips. The post attracted considerable attention because it touches on a core contradiction in on-device AI deployment: there is often a gap between advanced model architectures and the hardware characteristics of mobile chips.

What Is BiRefNet and Why Does It Struggle on Mobile?

BiRefNet is a model for high-resolution image segmentation, known for its excellent edge details and handling of transparent objects. Its architecture makes heavy use of deformable convolutions, an operation that allows the convolution kernel to have spatial adaptivity during sampling, thereby better capturing irregular shapes. However, deformable convolution is precisely a "nightmare" for mobile NPUs and GPUs.

Unlike standard convolution, deformable convolution requires dynamic computation of sampling positions, which involves additional offset prediction and bilinear interpolation steps. These operations can still be executed efficiently on desktop-grade GPUs, but on mobile NPUs, due to the lack of hardware acceleration support for dynamic indexing and interpolation operations, operators often cannot be mapped to the NPU's instruction set, resulting in compilation failures. The developer mentioned that "compilation to the NPU failed entirely," which is very likely due to unsupported operators.

Deformable Convolution was first proposed by Dai et al. in 2017 (Deformable ConvNets). Its core idea is to learn an additional two-dimensional offset for each sampling point on the regular sampling grid of standard convolution, enabling the convolution kernel to adaptively focus on the irregular geometric structures of the target. In the specific implementation, the offset is predicted from the input feature map by an additional convolution layer, and the sampling positions then become non-integer coordinates, so bilinear interpolation is needed to obtain the values at the corresponding positions from the feature map.

This mechanism significantly enhances the model's ability to model deformation, rotation, and non-rigid objects, often bringing accuracy improvements in tasks such as segmentation and detection. But the cost is the introduction of data-dependent dynamic indexing and interpolation operations, which breaks the static computation graph characteristics of standard convolution. For mobile inference frameworks that rely on fixed operator mapping and static scheduling compilation, such dynamic operators often cannot be effectively graphed or hardware-accelerated, and are recognized as high-risk operators in on-device deployment.

Operator Support Limitations of Qualcomm NPUs

Qualcomm's Hexagon NPU (and earlier DSPs) have relatively limited operator support, mainly optimizing static graph operators such as regular convolution, pooling, and activation. For dynamic operators such as deformable convolution and grid_sample, execution usually needs to fall back to CPU or GPU, or developers must manually implement approximate versions. The developer tried to compile BiRefNet directly to the NPU, but the toolchain (such as Qualcomm SNPE or QNN) may not recognize these complex operators, causing errors at the compilation stage.

Even if compilation passes, the NPU's parallel computing architecture is better suited to regular data flows, and the irregular memory access caused by deformable convolution will severely reduce utilization. Therefore, achieving sub-second latency on the NPU currently seems extremely challenging.

Qualcomm SNPE and QNN are two generations of on-device inference toolchains provided by Qualcomm. SNPE (Snapdragon Neural Processing Engine) is the earlier SDK, supporting conversion of models such as Caffe, TensorFlow, and ONNX and scheduling them to CPU, GPU (Adreno), or DSP/NPU (Hexagon) for execution; QNN (Qualcomm AI Engine Direct) is a newer low-level API that provides finer-grained backend control and is gradually replacing SNPE as Qualcomm's main deployment path for on-device AI.

What both have in common is their "operator whitelist + graph partitioning" approach: the toolchain checks one by one whether the operators in the model are supported by the target backend, and unsupported operators are partitioned out to fall back to CPU execution, or directly cause errors during the conversion/compilation stage. Therefore, when a model contains non-standard operators such as deformable convolution and grid_sample, a common phenomenon is compilation failure, or the generation of many fragmented subgraphs that lose the NPU acceleration benefits. Developers usually need to use the toolchain's operator analysis function to identify these "breakpoints" in advance.

Performance on GPU and Optimization Space

Compared with NPUs, mobile GPUs (such as the Adreno series) have somewhat better support for deformable convolution, because GPU shader programs can flexibly implement interpolation and dynamic indexing. But mobile GPUs have limited compute power and bandwidth, and BiRefNet itself is computationally heavy, especially with high-resolution inputs, so sub-second latency is still difficult to achieve. The developer did not disclose specific latency data, but said they "want sub-second performance," implying that current GPU performance may still be some distance from the goal.

Some possible optimization directions include: reducing input resolution, using model quantization (such as FP16 or INT8), replacing deformable convolution with lightweight alternatives (such as depthwise separable convolution plus spatial attention), or using knowledge distillation to train a smaller student model. But these changes may affect segmentation accuracy and require trade-offs.

Model quantization is one of the most commonly used compression methods in on-device deployment. FP16 quantization reduces weights and activations from 32-bit floating point to 16-bit, usually with very small accuracy loss, and modern mobile GPUs generally support FP16 computation, yielding significant bandwidth and compute benefits. INT8 quantization further maps values to 8-bit integers, requiring calibration datasets to estimate the dynamic range of activations and potentially introducing quantization errors; tasks sensitive to segmentation edge details need careful evaluation.

It is worth noting that bilinear interpolation and dynamic offset prediction in deformable convolution are relatively sensitive to quantization, and low-bit quantization may amplify interpolation errors. In practice, mixed-precision strategies are often used: keeping offset prediction and interpolation-related layers in FP16 while using INT8 for the rest, to strike a balance between latency and accuracy.

Are There Successful Community Cases?

As of the time of posting, the developer had not received a clear reply. In public information, there are also few cases of successful deployment of BiRefNet on mobile NPUs. This reflects a reality: many cutting-edge models prioritize accuracy in their design and are not sufficiently friendly to on-device deployment. However, there are already some segmentation models optimized for mobile devices in the community (such as MobileSAM and FastSAM), which achieve real-time performance by simplifying the architecture or adopting more efficient operators, and may serve as alternatives or references for improvement.

For scenarios where BiRefNet must be used, another approach is hybrid execution: running the deformable convolution part on the GPU and the rest on the NPU, but this introduces data transfer overhead and requires careful pipeline design.

Advice for Developers

If you face similar problems, you can try the following steps: first, use Qualcomm's QNN toolchain to analyze the model and confirm which operators are unsupported; second, consider ONNX Runtime's mobile deployment solution, which may provide an operator fallback mechanism; third, follow BiRefNet's official repository or community branches to see if there are mobile-optimized versions; finally, if sub-second latency is a hard requirement, you may need to re-evaluate model selection and choose a segmentation network designed specifically for mobile devices.

On-device AI deployment is never a simple model conversion; it requires a deep understanding of the match between hardware characteristics and model structure. The BiRefNet case reminds us that while pursuing accuracy, deployment friendliness is also worth attention.

One-line Summary

BiRefNet is difficult to deploy to mobile NPUs/GPUs due to deformable convolution, and the sub-second latency goal faces dual challenges of hardware and operator adaptation.

Paragraph Summary

The article revolves around a developer's Reddit request for help after failing to deploy BiRefNet to Qualcomm mobile GPUs and NPUs. BiRefNet excels at high-resolution segmentation and edge details, but its heavy use of deformable convolution requires dynamic offset prediction and bilinear interpolation, which are dynamic operators. Qualcomm Hexagon NPU has limited operator support, and the toolchain errors out at the compilation stage; although mobile GPUs can flexibly implement interpolation and dynamic indexing, they are constrained by compute power and bandwidth, making sub-second latency difficult to achieve. The article outlines optimization directions such as quantization, resolution reduction, operator replacement, knowledge distillation, and GPU/NPU hybrid execution, along with their accuracy trade-offs. It notes that there are no public cases of successful BiRefNet deployment on mobile NPUs in the community, and suggests developers first use QNN to analyze operators, consider ONNX Runtime fallback mechanisms, and re-evaluate model selection if necessary.