The First Camera-Native Stereo VR180 Dataset: Filling the Gap in Immersive Video Research

Why VR180 Research Needs a "Clean" Dataset

Immersive VR180 video is increasingly captured with professional stereo fisheye cameras, yet a long-overlooked problem persists in publicly available research resources: nearly all public VR180 data originates from online platforms like YouTube. These materials have already been processed through unknown stitching, projection, and compression pipelines, and lack lens calibration information. In other words, researchers receive a "finished product" rather than "raw material," making it impossible to study the relationships between encoding, projection, and quality from the ground up.

A newly published arXiv paper (arXiv:2610.10607v1) addresses exactly this pain point. The research team presents a stereo VR180 dataset captured directly by cameras, recorded using two Blackmagic URSA Cine Immersive cameras — providing the academic community with native footage untouched by third-party processing for the first time.

What's Actually Inside the Dataset

The dataset contains 1,211 samples, including 636 stereo video clips (totaling 2,220.8 seconds, mostly at 90fps) and 575 stereo still images. Each sample is released simultaneously in multiple formats, covering the complete pipeline from raw capture to usable output:

• Camera-native Blackmagic RAW: the closest to raw sensor output;
• Split-eye native fisheye HEVC: 8160×7200 resolution per eye;
• Half-equirectangular projection HEVC: 7200×7200 resolution per eye.

Beyond the footage itself, the dataset includes factory lens calibration parameters, portable fisheye/half-equirectangular projection conversion tools, and AI-generated annotations for scenes and visual challenges. This means researchers get not just the visuals, but also the geometric properties of the lenses, along with tools to freely convert between projection formats.

This "fully open pipeline" approach gives the dataset the foundational conditions needed for research into compression, projection, viewport-adaptive streaming, and more — rather than locking researchers into a single pre-processed format.

Half-Equirectangular Projection (Half-ERP) is one of the standard display formats for VR180 content. Standard equirectangular projection maps the full 360° sphere to a rectangular image, but since VR180 only covers a 180° forward field of view, the "half" version suffices. Compared to the fisheye format, half-equirectangular projection distributes pixels more uniformly across the image with less edge distortion, and is more compatible with the block-partitioning structure of traditional video codecs like HEVC/H.265 — which is the geometric reason for its encoding efficiency advantage. Fisheye images, due to their nonlinear radial projection, accumulate redundant pixels at the edges, forcing the encoder to spend more bitrate describing the texture in those distorted regions. This directly explains the ~38% bitrate gap observed in the experiments below.

A Key Finding: Native Fisheye Encoding Is More "Bitrate-Hungry"

The paper's most valuable experimental finding comes from an encoding comparison study. The research team re-encoded the released fisheye and half-equirectangular projection footage using x265, covering 24 clips, both eyes, four bitrate points, and nine viewing directions.

The results show that at equivalent viewport quality, native fisheye encoding requires higher bitrates than half-equirectangular projection encoding across all 24 clips, with a median difference of approximately 38% — and this gap holds across every region of the field of view.

This finding has practical implications. It demonstrates that in VR180 transmission and storage scenarios, the choice of projection format directly affects bandwidth efficiency. Half-equirectangular projection holds a clear advantage in encoding efficiency, providing quantitative evidence for format decisions on streaming platforms. That said, the fisheye format preserves geometric information closer to the original capture, and whether trading encoding efficiency for this advantage is worthwhile remains a tradeoff that depends on the specific application.

"Viewport quality" is a specific concept in VR video quality assessment, distinct from traditional full-frame PSNR or SSIM metrics. Since viewers of VR content only see a portion of the image at any given moment (i.e., the viewport corresponding to their current head orientation), quality evaluation must simulate the user's actual viewing region rather than averaging across the full frame. This experiment covers nine viewing directions, meaning multiple representative viewport positions were uniformly sampled across horizontal and vertical axes, ensuring the conclusions are not dependent on any particular orientation and carry broad generalizability. This evaluation approach is closer to real user experience than full-frame metrics, and represents the mainstream methodology in current immersive video quality research.

Implications for Industry and Research

VR180 research has long been constrained by the "second-hand" nature of its data sources, with many conclusions about encoding and projection distortion contaminated by confounding variables introduced by platform processing pipelines. The availability of this camera-native dataset allows researchers, for the first time, to isolate these variables under controlled conditions.

For content producers, the factory lens calibration and projection conversion tools lower the barrier to working with professional stereo footage. For algorithm researchers, the AI-annotated scene and visual challenge information provides ready-made references for training and evaluation.

The dataset is publicly available on Hugging Face (lulinxuan/VR180), with companion tools hosted on GitHub (lulinxuan/vr180-dataset-tools), allowing anyone to access and reproduce the experiments directly. As immersive video hardware becomes increasingly widespread, open resources grounded in the capture source like this one could become critical infrastructure for advancing the rigor of VR180 encoding and projection research.