Attention Distribution
A continuous distribution centered on the fixation point that decays with spatial distance, used to model the allocation of human visual attention resources and injectable into models as an inductive bias.
Timeline (last 90 days)
A new work models visual attention as a continuous distribution centered on the current fixation point that decays with spatial distance, and encodes this distribution into reward model training as an additional supervisory signal.
Whether the parameterization of the attention distribution applies to non-reading tasks remains an open question.
Placing system prompts and the current task goal at the beginning and end of the context window is an engineering strategy with empirical backing, because models pay more attention to the beginning and end content.
All Facts (3)
Whether the parameterization of the attention distribution applies to non-reading tasks remains an open question.
50%UnverifiedPlacing system prompts and the current task goal at the beginning and end of the context window is an engineering strategy with empirical backing, because models pay more attention to the beginning and end content.
50%UnverifiedA new work models visual attention as a continuous distribution centered on the current fixation point that decays with spatial distance, and encodes this distribution into reward model training as an additional supervisory signal.
50%