0.85GB Dancing Stick Figures Dataset: Train a Video Generation Model in One Hour on Free Colab

Where Does the Real Barrier to Video Generation Lie?

Video generation has long been one of the highest-barrier areas in deep learning. The most immediate obstacle isn't the algorithms themselves — it's data scale. Training a video generation model typically requires datasets ranging from tens to hundreds of gigabytes, putting the whole endeavor out of reach for most students, independent developers, and beginners. Downloading the data is painful, training is expensive, and running a complete experiment on a free GPU seems nearly impossible.

Recently, developer sprited-ai shared a clever approach on Reddit: instead of letting people get lost in massive real-world datasets, why not build a deliberately small teaching dataset that lets learners download, understand, and train through an entire workflow within a single Colab session? The project is called Dancing Stick Figures.

The core value of this approach is reducing cognitive load. When the dataset is only 0.85GB and the content is simple stick figure dancing animations, learners can focus on model architecture, training dynamics, and evaluation methods — rather than getting bogged down by data engineering and compute constraints.

Core Design of the Dancing Stick Figures Dataset

Scale and Dual-Configuration Approach

Dancing Stick Figures is remarkably restrained and thoughtfully designed, containing:

• 1,430 six-second short video clips totaling 514,800 annotated frames
• A 64×64 mini configuration at just 0.85 GB, designed specifically for free Colab
• A 128×128 full configuration that preserves richer detail for more advanced experiments

This dual-configuration design is clever: the mini version is small enough to fit within a free Colab T4's VRAM and session time; the full version gives users with more compute room to explore further.

Rich Multi-Task Annotation

Despite simple visuals, the annotations are quite comprehensive. The full dataset includes:

• Precise 2D and 3D positions plus visibility flags for 27 joint keypoints
• Depth maps, normal maps, and part segmentation maps
• Camera parameters and raw motion data

These annotations mean the dataset isn't limited to video generation — it can also support teaching experiments in pose estimation, depth prediction, semantic segmentation, and more. For a teaching dataset, this "small but complete" design means a single dataset can support multiple assignments across an entire course.

Lightweight Evaluation via Fixed Color Coding

One particularly noteworthy design detail: each limb maintains fixed color coding. This seems simple, but it enables a small NumPy-based scorer to automatically detect "missing limbs" or "disconnected limbs" in generated outputs. This rule-based lightweight scoring approach avoids the need for complex evaluation models, letting beginners intuitively understand exactly where the model went wrong.

Run the Full Training Pipeline on Free Colab in One Hour

The author emphasizes that the dataset itself is the main release, but to demonstrate its practicality, he also provides: a free Colab notebook, several small reference baseline models, pretrained checkpoints, and the aforementioned scorer.

The reference Colab takes approximately one hour on a T4 GPU to complete the entire workflow:

1. Train an image baseline model
2. Use the image model to warm-start an eight-frame video baseline model
3. Generate a final 5.6-second inference rollout

An interesting observation from this experiment: the warm-started baseline model reaches the same loss level in roughly 4,000 steps that a cold-start model achieves at 10,000 steps. This concretely demonstrates the efficiency advantage of transfer learning in video generation — learning to draw good single frames first, then learning to animate them, is far more efficient than trying to learn both simultaneously from scratch.

Project Scope: Not a Video Model, but a Learning Tool

The author is refreshingly honest about the project's boundaries: "This is not a finished video model. It's a small dataset for building, breaking, and understanding things yourself."

This captures the entire project's design philosophy. In an era dominated by the LLM arms race, people are conditioned to chase SOTA performance and parameter counts — often overlooking the value of genuine understanding. Dancing Stick Figures takes the opposite approach: it doesn't aim to generate realistic video, but rather to let learners see the full mechanics of video generation clearly within a single experiment: how data is organized, how a model extends from images to video, how loss decreases, and how generated results are evaluated.

The license choices also reflect the author's open spirit: data is released under CC0 (full copyright waiver), code under MIT. This means anyone — especially educators and students — can use it in courses and projects without any legal friction.

Implications for AI Education and Video Generation Onboarding

At the end of the post, the author asks the community: for classroom teaching or a first project, what would be most helpful — shorter notebooks, assignment design ideas, pose estimation baselines, or more varied motion types?

This open-ended question itself reflects a trend that's gaining serious attention: the "accessibility" of AI educational resources is becoming just as important as the technical frontier. As foundation models and frameworks continue to mature, helping the next generation of developers truly understand underlying principles — rather than just calling APIs — becomes a critical challenge.

A thoughtfully designed small dataset, paired with a complete workflow that runs in one hour, may do more to build a beginner's intuition for video generation than any sophisticated research paper. From this perspective, Dancing Stick Figures matters not for what it generates, but for making video generation tangible, decomposable, breakable, and fixable.

Resources and Getting Started

For readers who want to try it out, the complete project resources are:

• Colab Notebook: Run directly in your browser with one click — no local setup required
• Code Repository: sprited-ai/dancing-stick-figures on GitHub (MIT license)
• Dataset Download: sprited/dancing-stick-figures on Hugging Face (CC0 license)
• Pretrained Checkpoints: sprited/dancing-stick-figures-baselines on Hugging Face

Whether you're an AI educator, a student looking to break into video generation, or an independent developer working with limited compute, Dancing Stick Figures is a low-barrier, thoughtfully designed starting point.