Single-Camera Boxing Analysis System: Building an Automated Sports Data Pipeline with SAM 3

Extracting Complete Boxing Data from a Single Camera Feed

A developer spent several months building an automated boxing event analysis pipeline using a "vibe-coding" approach. The goal is straightforward: given only footage from a single camera, the system automatically tracks fighters, recognizes punches, reconstructs top-down ring positions, and estimates real-world distances between athletes. This kind of work typically requires multi-camera setups, professional sensors, or even wearable devices — yet this project attempts to accomplish everything from ordinary video input.

Boxing has long been a difficult sport to quantify. Fighters move rapidly within a small ring, frequently obscuring each other, and hand movements are fast enough to cause most tracking algorithms to lose their targets. Reliably extracting structured data from a single, potentially handheld, shaky camera requires a carefully coordinated stack of computer vision modules working in concert.

Technical Stack Breakdown: A Pipeline of Multiple Vision Models

According to the author's description, this pipeline is assembled from multiple cutting-edge vision models, each responsible for solving a specific sub-problem.

Fighter Tracking and Identity Preservation

Core object segmentation and tracking is handled by SAM 3 (Segment Anything Model), which not only identifies the fighters but also extracts the ring canvas and ropes as separate pixel-level masks. This step essentially creates a structured spatial reference frame for the entire scene.

In a boxing match, two fighters frequently clinch and obscure each other, making it easy for standard trackers to swap identities after close contact. To address this, the author incorporates OSNet for person re-identification (re-ID), allowing the system to maintain each fighter's identity label based on appearance features even after they reappear from behind occlusion.

SAM (Segment Anything Model) is a general-purpose image segmentation foundation model released by Meta AI in 2023. Its core capability is generating pixel-level segmentation masks for virtually any object in an image with near-zero shot prompting. SAM 3 is an iterative version with enhancements in video tracking and 3D perception. Unlike traditional object detection, which only outputs bounding boxes, pixel-level masks precisely describe which pixel belongs to which object — particularly valuable in a ring setting. Masks can accurately outline each fighter's silhouette while also extracting the canvas floor and ropes as independent spatial references, providing the geometric foundation for subsequent coordinate mapping.

OSNet (Omni-Scale Network) is a lightweight neural network designed specifically for person re-identification tasks, capable of simultaneously capturing fine-grained appearance features (such as clothing color and texture details) at multiple scales. The core challenge of re-ID is this: when the same person briefly disappears from the camera's view and reappears, can the system correctly associate them with their previous trajectory? In boxing, body overlap during clinches causes trackers to "lose their way." The appearance features provided by OSNet act like a visual ID card for each fighter, enabling the system to restore correct identity bindings once occlusion clears.

Punch Detection and Classification

The action recognition stage uses RTMPose for pose estimation, followed by a lightweight classifier that detects punches — not just which hand threw the punch, but also the type. This means the system can theoretically tally the frequency and distribution of different punch types like jabs, hooks, and uppercuts, rather than simply counting punches.

RTMPose is a real-time multi-person pose estimation framework developed by OpenMMLab, capable of outputting human keypoint coordinates at high frame rates — including pixel positions of joints like shoulders, elbows, and wrists. In boxing, the displacement trajectory, velocity vector, and relative position of the wrist joint to the torso are the core features for distinguishing punch types: a jab is typically a quick, linear extension of the lead hand; a hook follows an arcing trajectory with a bent elbow; and an uppercut shows a vertical upward acceleration. The lightweight classifier performs secondary inference on RTMPose's temporal keypoint sequences, mapping kinematic patterns to specific punch type labels. This two-stage "pose estimation + sequence classification" design strikes a good balance between computational efficiency and accuracy, and represents one of the dominant paradigms in sports action recognition today.

Spatial Positioning: From Image Pixels to Ring Coordinates

Action recognition alone isn't enough — the other half of boxing analytics lies in spatial information: where fighters are positioned on the ring, how far apart they are, and how they move and maneuver.

The author uses the previously extracted ring masks to fit a ring model, mapping pixel positions in the frame to a top-down view coordinate system. This is similar to the tactical board view common in soccer analytics, transforming flat video into quantifiable spatial data.

To address the challenge of single-camera and especially handheld footage, the pipeline incorporates camera-motion compensation to counteract coordinate drift caused by camera shake, ensuring output data isn't corrupted by lens movement. This is a critical component for making a single-camera solution practically viable.

Going a step further, the author also integrates SAM 3D Body to add metric real-world distance estimation between the two fighters. Compared to purely 2D pixel distances, 3D measurements with scale information provide a more reliable numerical basis for tactical assessments like "close-quarters clinching" or "mid-to-long range exchanges."

Recovering top-down coordinates from an obliquely shot single-camera feed is fundamentally a Homography problem: finding a 3×3 projection matrix that maps pixel coordinates in the camera plane to real-world coordinates in the ring plane. This requires identifying known geometric features such as the ring's four corner points or rope boundaries — which the ring masks extracted by SAM 3 conveniently provide. Camera motion compensation is typically achieved by comparing feature point displacements between adjacent frames (using optical flow or RANSAC-based stable estimation) to back-calculate the camera's translation and rotation, then subtracting this jitter from the target's motion trajectory. Without this compensation, the several-hertz oscillations from handheld shooting would manifest as false drift in fighter trajectories on the top-down coordinate map, completely masking actual movement strategies.

Why This Project Is Worth Attention

The significance of this approach lies not in any single technical breakthrough, but in what it demonstrates about the engineering potential of combining today's open-source vision models. SAM 3, OSNet, RTMPose, and SAM 3D Body are each individually mature, but chaining them into an end-to-end pipeline capable of handling real sports scenarios — resisting occlusion and camera shake — requires extensive tuning and engineering trade-offs.

From an application perspective, a low-cost single-camera analysis solution is highly appealing for amateur boxing, training gyms, and content creators alike. No expensive multi-camera systems are needed; an ordinary match recording can produce near-professional tactical data. This "lowering the barrier to entry" direction is exactly how computer vision is finding its way into sports applications.

That said, the author self-describes this as built through "vibe-coding," meaning it's more of an exploratory personal project than a rigorously validated product. Key metrics — punch classification accuracy, top-down mapping error, reliability of distance estimation — are not quantitatively evaluated in the original post, and actual performance still needs more public data to substantiate.

Summary

This project chains multiple open-source vision models into a functional boxing analysis pipeline covering tracking, identity preservation, action recognition, spatial positioning, and distance measurement. Its value lies primarily in the engineering approach: using off-the-shelf models to tackle complex real-world sports analysis challenges, with targeted solutions for practical constraints like single cameras and handheld shake. For developers interested in deploying computer vision in sports contexts, this is a valuable reference for model composition.