FilterFusion with 109K Parameters: A New Approach to Real-Time Video Denoising on a Single CPU Thread

Video denoising has long faced an awkward triangular dilemma: image quality, speed, and compute cost—it's hard to have all three at once. Pure deep models offer high accuracy but come with staggering computational costs, while classical filters are efficient but lack adaptability. A new work called FilterFusion, from an RSS source, attempts to break this deadlock—achieving 55.9 frames per second at 1080p on a single CPU thread with just 109K parameters.

Background: The Real-Time Denoising Challenge Under Mixed Degradation

Real-world video denoising in communication systems never deals with a single type of noise. Sensor noise, compression artifacts, and motion-induced temporal inconsistency often coexist, stacking together into complex mixed degradation.

Pure deep learning methods can achieve high accuracy metrics in such scenarios, but their inference cost is extremely high—some lightweight CNN baselines take over 2 seconds to process a single frame, making them completely unusable in real-time communication scenarios. Classical filtering methods (Gaussian blur, median filtering, etc.) are exactly the opposite: computationally simple and robust, but lacking adaptability when facing varying noise types, unable to dynamically adjust based on content.

FilterFusion's approach is to take the best of both: use learned temporal features to provide "prior guidance" and use an interpretable filter bank to perform the actual synthesis.

PSNR and SSIM are the two most commonly used quality evaluation metrics in the video/image denoising field. PSNR (Peak Signal-to-Noise Ratio) measures the degree of distortion by calculating the mean squared error between the processed image and a clean reference image, expressed in decibels (dB), where a higher value indicates smaller pixel-level error. SSIM (Structural Similarity Index) evaluates the perceptual similarity of two images across three dimensions—luminance, contrast, and structure—with values ranging from 0 to 1, where closer to 1 indicates better structural preservation. The two are complementary: PSNR focuses on per-pixel accuracy, while SSIM better reflects the human eye's subjective perception of structural information. Reporting both simultaneously in video denoising papers is standard practice for determining whether a method meets standards in both fidelity and visual quality.

CRVD (Captured Raw Video Denoising) is a video denoising dataset oriented toward real sensor noise. Unlike common synthetic noise datasets, CRVD uses professional equipment to synchronously capture noisy videos and clean references in real scenes, where the noise comes from the physical characteristics of the sensor itself rather than artificially added Gaussian noise. Therefore, it better reflects the mixed degradation of sensor noise, compression artifacts, and motion inconsistency in communication systems, and has become an important benchmark for evaluating the practical deployment capability of denoising methods.

Core Architecture: Low-Resolution Controller + Full-Resolution Filter Bank

FilterFusion's overall design revolves around one core principle—make expensive decisions at low resolution, and perform cheap synthesis at original resolution.

Low-Resolution Controller

The model first predicts a low-cost guidance signal, which is generated in low-resolution space and is responsible for deciding how to blend different denoising branches. The controller explicitly injects two types of prior cues:

• Temporal variance: reflects inconsistency caused by inter-frame motion
• Spatial variance: reflects the spatial distribution characteristics of noise within a single frame

Injecting these two types of variance cues into the low-resolution controller means that when making blending decisions, the model already "knows" which regions have intense motion and which regions have heavier noise, thereby allocating denoising strength in a targeted manner.

Interpretable Filter Bank

The synthesis path uses a compact filter bank containing classical operations such as Gaussian blur, median filtering, and weighted summation. These operations are characterized by being simple, regular, and highly predictable—there are no complex operators from convolutional neural networks, which also leaves ample room for subsequent engineering optimization.

Each denoising branch produces a response, and the controller adaptively blends these responses based on the guidance signal. The entire network has only 109K parameters, and the network's own computational cost is only 0.18 GFLOPs.

GFLOPs (billion floating-point operations) is a commonly used unit for measuring the inference compute of neural networks. One FLOP refers to one floating-point addition or multiplication, and the total number of operations required for a model to process one frame of image is its computational cost. 0.18 GFLOPs means the network requires only about 180 million floating-point operations to process a single frame, which is two to three orders of magnitude lower than mainstream deep models that often require tens or even hundreds of GFLOPs. Parameter count (109K) and GFLOPs are two different dimensions: the former measures model storage footprint, while the latter measures runtime computational overhead. FilterFusion is extremely low in both, which is what makes reaching 55.9 FPS on a pure single CPU thread possible—this is the direct benefit brought by the design of a "compilable synthesis path."

Performance Data: 34.72 dB PSNR and 55.9 FPS Achieved Simultaneously

Test results on CRVD (a real sensor noise dataset) demonstrate the practical effectiveness of this design:

Metric	Value
PSNR	34.72 dB
SSIM	0.9330
Network compute	0.18 GFLOPs
Parameter count	109K
CPU inference speed (1080p, single thread)	17.9ms/frame (55.9 FPS)

The most noteworthy column is inference speed. After implementing the synthesis path in compiled C code, the research team achieved 17.9ms per frame at 1080p resolution on a single CPU thread, or 55.9 FPS—not only exceeding the real-time threshold (typically 30 FPS), but also 122 to 252 times faster than lightweight CNN baselines that require more than 2 seconds to process a single frame.

The direct source of this speed advantage is the simplicity and regularity of the filter bank operations. Operations such as Gaussian blur and median filtering can execute efficiently after compilation, without relying on GPU acceleration or specialized inference frameworks.

Design Philosophy: Low-Resolution Control + Explicit Filter Synthesis

FilterFusion's contribution is not just a set of numbers, but rather its validation of the feasibility of an engineering path: driving explicit filter synthesis through low-resolution control can achieve a practical balance among quality, efficiency, and speed.

The insight of this path is that not all decisions need to be made at original resolution. Assigning high-level decisions such as "which filters to use and in what proportions to blend them" to a low-resolution controller, and assigning "actually executing filtering" to efficient classical operators, preserves the adaptability of learning while avoiding the computational redundancy of end-to-end deep models.

For engineering teams that need to deploy real-time video denoising in communication systems, edge devices, or CPU environments, this "compilable synthesis path" means the model can be directly deployed on existing infrastructure without relying on GPU inference stacks.

Of course, this method has currently been validated mainly on the CRVD dataset, and whether the filter bank remains robust in more complex real-world scenarios (such as intense motion or multi-source noise stacking) still requires further observation.

One-line Summary

FilterFusion achieves real-time 1080p video denoising with 109K parameters, reaching 55.9 FPS on a single CPU thread.

Paragraph Summary

To address the difficulty of balancing image quality, speed, and compute in real-world video denoising, FilterFusion proposes an architecture combining low-resolution control with full-resolution explicit filter synthesis. The model generates a guidance signal in low-resolution space, explicitly injects temporal and spatial variance priors, and adaptively blends the outputs of a classical filter bank such as Gaussian blur and median filtering. The entire network has only 109K parameters and 0.18 GFLOPs, achieving 34.72 dB PSNR and 0.9330 SSIM on the CRVD dataset. After its synthesis path is implemented in compiled C code, it reaches 17.9ms per frame, or 55.9 FPS, at 1080p on a single CPU thread, 122 to 252 times faster than lightweight CNN baselines. This work validates the feasibility of the engineering path of low-resolution control driving explicit filter synthesis for real-time video denoising, offering new ideas for deployment on edge devices and CPU environments, though its robustness in more complex real-world scenarios still needs further validation.