Knowledge Transfer Between Simple Discrete Worlds: A Study of Generalization in Deep Recursive Models

A fundamental capability of general intelligence is the ability to adapt to new environments and correctly reason about what outcomes are possible when faced with rules never encountered before. A new study systematically investigates this question by constructing small discrete worlds, offering new approaches for evaluating deep recursive reasoning models.

Research Framework: Sudoku-Style Puzzles and Rule Composition

According to the study, the experiments employed an elegant design: using Sudoku-style puzzles constructed from shared base rules plus a small number of additional rules, forming a set of grid configurations consistent with a common rule set. Researchers trained a deep recursive model on various rule combinations, then tested its performance at four different levels of novelty.

These four levels were: entirely new puzzles, new combinations of known rules, rule pairings that had never appeared together, and rule instances completely unseen during training. The model received active rules in two ways: symbolic descriptions or two solved examples. Additionally, a model receiving no rule information served as a baseline control.

This design allowed researchers to test multiple generalization scenarios in a more systematic manner than benchmarks like ARC-AGI, particularly opening new ground for evaluating deep recursive reasoning models—which are typically tested only on saturated Sudoku benchmarks.

Deep recursive reasoning models refer to a class of neural network architectures that progressively refine internal representations through multiple iterations of forward computation (i.e., "recursion" or "looping"). Unlike traditional feedforward models, these models repeatedly apply the same set of parameters to intermediate states during inference, allowing computational depth to grow adaptively with problem complexity, thereby possessing the capability to solve structured tasks such as combinatorial optimization and logical reasoning. Typical implementations include recurrent Transformers, Universal Transformers, and variants like Neural Turing Machines. They excel at constraint satisfaction problems such as Sudoku and mazes, but prior evaluations have been largely confined to saturated benchmarks with fixed rules, making it difficult to determine their generalization boundaries.

ARC-AGI (Abstraction and Reasoning Corpus) is a general intelligence benchmark proposed by François Chollet, consisting of a series of colored grid transformation tasks that require models to infer abstract transformation rules from very few examples and apply them to new inputs. Its original purpose was to measure "sample-efficient generalization" capability rather than reliance on large-scale pretraining. However, ARC-AGI's task rules are often difficult to formalize, and its novelty dimensions are not sufficiently controllable. Therefore, this study adopts Sudoku-style discrete worlds as a complement, enabling more systematic separation of different generalization challenges such as "new rules," "new combinations," and "new instances."

Core Findings: The Tension Between Symbolic Descriptions and Examples

The results revealed a thought-provoking pattern. When recombining rules seen during training, symbolic descriptions matched or even exceeded the performance of examples—achieving 99% versus 95% accuracy on new combinations. This suggests that when models face rearrangements of known rules, explicit symbolic rule specifications can provide more precise guidance.

However, when facing rule instances that had never co-occurred during training, the situation reversed. Symbolic descriptions dropped below the no-information baseline, while the example-based approach remained above baseline. This finding suggests that symbolic descriptions may become misleading when handling completely unfamiliar rule combinations, whereas concrete examples demonstrate greater robustness.

The reasons behind this discrepancy merit deeper consideration. Symbolic descriptions require models to understand abstract rules and apply them to new situations; when rule components have never co-occurred, models may over-rely on prior associations formed during training, leading to negative transfer. Examples, by contrast, provide more concrete and contextualized information, enabling models to extract transferable patterns from actual demonstrations.

Negative transfer is a phenomenon in transfer learning where knowledge or priors learned on a source task actually harm performance when applied to a target task. When symbolic descriptions contain rule components that never co-occurred during training, the model may forcibly apply existing rule association patterns, leading to incorrect inferences about the current task. This is opposed to positive transfer, which refers to old knowledge facilitating new task learning. Negative transfer is particularly common in compositional generalization scenarios because models tend to take "shortcuts," relying on surface co-occurrence statistics rather than genuine rule understanding. The study's observation of symbolic description accuracy falling below the no-information baseline is a typical manifestation of negative transfer.

No-information baseline refers to the performance level of a model receiving no rule information whatsoever, typically approximating random guessing or reliance on training set biases. It measures the model's inherent bias under zero rule prompting. In this study's experiments, when symbolic descriptions failed due to negative transfer, their accuracy fell even below this baseline, indicating that incorrect rule prompts are worse than no prompts at all.

Implications for General Intelligence Research

The significance of this study extends beyond Sudoku puzzles themselves. It provides a systematic framework for studying how knowledge transfers between discrete worlds and where models' generalization boundaries lie when facing different degrees of novelty.

Compared to benchmarks like ARC-AGI, this framework allows finer-grained control over the dimensions of novelty—whether it is the rules themselves that are new, the rule combinations that are new, or the rule instances that are new. Such fine-grained control is crucial for understanding models' generalization mechanisms.

Deep recursive reasoning models are typically evaluated only on saturated benchmarks like Sudoku, which limits judgment of their true generalization capabilities. The method proposed in this study provides new ground for evaluating such models, enabling researchers to observe under controlled conditions how models cope with different levels of unfamiliarity.

The research also hints at a deeper question: under what conditions do abstract rules facilitate knowledge transfer better than concrete examples? The answer appears to depend on the nature of the novelty. When novelty lies in the recombination of rules, abstract descriptions prevail; when novelty involves components that have never co-occurred, concrete examples are more reliable. This insight has direct guiding significance for designing AI systems capable of adapting to new environments.

One-line Summary

A study uses Sudoku-style discrete worlds to test knowledge transfer in deep recursive models, finding that the advantage of abstract rules versus concrete examples depends on the type of novelty.

Paragraph Summary

A new study systematically investigates the knowledge transfer capabilities of deep recursive reasoning models through Sudoku-style discrete worlds. Experiments test models at four levels of novelty: entirely new puzzles, new combinations of known rules, rule pairings that never co-occurred, and completely unseen rule instances, with rules provided either as symbolic descriptions or solved examples. Results show that when recombining known rules, symbolic descriptions achieve 99% accuracy, surpassing examples at 95%; but when facing rule instances that never co-occurred, symbolic descriptions drop below the no-information baseline while examples remain above it. This indicates that abstract rules excel in compositional generalization, while concrete examples are more robust when handling entirely new components. This framework enables finer control over novelty dimensions than benchmarks like ARC-AGI, offering new approaches for evaluating deep recursive models.