SwitchSplat
A framework for modeling discrete object states based on 3D Gaussian Splatting (3DGS). Through a base-plus-slot decomposition and a differentiable assignment mechanism, it learns binary object states from casually captured videos and supports free composition of states at inference time for novel view synthesis.
Timeline (last 90 days)
SwitchSplat proposes a base+slot decomposition architecture, using a shared static base to capture the invariant part, with each slot independently holding two binary states of an interactable object.
In SwitchSplat, the process of assigning Gaussian points to slots is fully differentiable, allowing the model to directly discover the geometry of each state from video without manual state mask annotation.
At inference time, SwitchSplat supports freely combining slot states and rendering from arbitrary viewpoints, including state combinations that never appeared during capture.
SwitchSplat is validated on a new casually captured dataset and outperforms static and dynamic baseline methods in novel view synthesis quality.
SwitchSplat is the first work to extend 3DGS to discrete state combinations.
The current SwitchSplat method focuses on binary states (on/off), and objects with multi-state or continuously varying states may require an extended slot design.
Occlusion and motion blur in casually captured videos may affect the quality of SwitchSplat's geometry learning.
The project page of SwitchSplat is https://switchsplat.github.io.
All Facts (8)
SwitchSplat proposes a base+slot decomposition architecture, using a shared static base to capture the invariant part, with each slot independently holding two binary states of an interactable object.
50%UnverifiedIn SwitchSplat, the process of assigning Gaussian points to slots is fully differentiable, allowing the model to directly discover the geometry of each state from video without manual state mask annotation.
50%UnverifiedAt inference time, SwitchSplat supports freely combining slot states and rendering from arbitrary viewpoints, including state combinations that never appeared during capture.
50%UnverifiedSwitchSplat is validated on a new casually captured dataset and outperforms static and dynamic baseline methods in novel view synthesis quality.
50%UnverifiedSwitchSplat is the first work to extend 3DGS to discrete state combinations.
50%UnverifiedThe current SwitchSplat method focuses on binary states (on/off), and objects with multi-state or continuously varying states may require an extended slot design.
50%UnverifiedOcclusion and motion blur in casually captured videos may affect the quality of SwitchSplat's geometry learning.
50%UnverifiedThe project page of SwitchSplat is https://switchsplat.github.io.
50%