AskStream
The first multi-turn streaming video understanding benchmark covering both video uncertainty and user uncertainty, comprising four task categories and eight subtasks, used to evaluate models' ability to ask questions proactively when information is incomplete.
Timeline (last 90 days)
The researchers propose treating "asking" as a third type of proactive action and introduce the first multi-turn benchmark, AskStream.
AskStream is the first multi-turn streaming video understanding benchmark covering both video uncertainty and user uncertainty.
The AskStream benchmark contains four task categories and eight subtasks, covers 438 videos, and includes manual annotation of 2,714 frame-level time windows.
Each frame-level time window in AskStream corresponds to three action types: wait, ask, and answer.
AskStream divides uncertainty into two categories: video uncertainty and user uncertainty.
The paper's analysis points out that when judging "when to ask," models tend to act too early or too late, and on "what to ask," they often pose irrelevant or redundant questions.
All Facts (6)
The researchers propose treating "asking" as a third type of proactive action and introduce the first multi-turn benchmark, AskStream.
50%UnverifiedAskStream is the first multi-turn streaming video understanding benchmark covering both video uncertainty and user uncertainty.
50%UnverifiedThe AskStream benchmark contains four task categories and eight subtasks, covers 438 videos, and includes manual annotation of 2,714 frame-level time windows.
50%UnverifiedEach frame-level time window in AskStream corresponds to three action types: wait, ask, and answer.
50%UnverifiedAskStream divides uncertainty into two categories: video uncertainty and user uncertainty.
50%UnverifiedThe paper's analysis points out that when judging "when to ask," models tend to act too early or too late, and on "what to ask," they often pose irrelevant or redundant questions.
50%