[KongchangAI]
Benchmark

SCAN

A benchmark for evaluating the systematic compositional generalization ability of neural network systems. It has been used to reveal shortcomings in the compositional capabilities of large language models.

Timeline (last 90 days)

Oct 2

Researchers have designed test sets such as SCAN, COGS, and ARC to probe compositional generalization ability.

Unverified50%
Sep 17

Benchmarks such as SCAN and COGS reveal clear shortcomings in large language models' systematic compositional abilities.

Unverified50%
Sep 11

The SCAN and LOOK algorithms originated in the disk scheduling domain and were later adopted in elevator control.

Unverified50%
Sep 11

The LOOK algorithm is an improved version of SCAN, where the elevator only travels to the last requested floor in the current direction before reversing, rather than being forced to run all the way to the top or bottom floor, thereby reducing unnecessary travel.

Unverified50%
Sep 11

SCAN, LOOK, and Nearest Car First algorithms are all single-objective greedy strategies, which are prone to elevator bunching in scenarios involving multiple elevators and non-uniform passenger traffic.

Unverified50%
Sep 8

Lake, Baroni, and others validated on benchmarks such as SCAN and COGS that current deep learning models have significant shortcomings in systematic generalization.

Unverified50%

All Facts (7)

Source Articles