The Road to AI in Autonomous Systems: From Individual Cognition to Collective Intelligence

Autonomous systems are increasingly regarded by researchers as the ultimate frontier of artificial intelligence. A new paper published on arXiv (arXiv:2609.30291) systematically explores how to bring AI into autonomous systems, arguing that achieving true autonomy requires combining connectionist AI with symbolic AI, while integrating AI techniques with systems engineering methods. The paper's central contribution is a comprehensive framework for designing and evaluating autonomous systems.

A General Agent Architecture: Long-Term Memory at the Core

The framework is built around a general agent architecture. The authors describe agent behavior as a combination of cognitive functions, all organized around a long-term memory that holds the agent's continuously evolving knowledge.

This design philosophy differs from many of today's purely data-driven AI systems. By placing "memory" at the center, agents don't simply react to inputs in the moment — they continuously accumulate and update their own knowledge, using it to guide future behavior. This knowledge-centric architecture embodies the fusion of symbolic tradition and connectionist capability: the perception layer can be driven by neural networks, while the decision-making and planning layer requires structured knowledge representation.

Connectionist AI, represented by neural networks, excels at learning patterns from large amounts of data and performs remarkably well on tasks like perception, language understanding, and image recognition — but its internal representations are hard to interpret and its reasoning process lacks transparency. Symbolic AI, based on logical rules and structured knowledge representation, enables interpretable deductive reasoning and planning, but struggles with the noise and uncertainty of the real world. These two paradigms have long been viewed as opposing camps — the debate between "neural network advocates" and "GOFAI (Good Old-Fashioned AI) advocates" has spanned decades of AI development. The idea of combining them is not new, but how to achieve complementary strengths in engineering practice remains an open challenge. A long-term memory-centric agent architecture is one concrete attempt to bridge these two paradigms: neural networks extract features from perceptual signals, while a symbolic knowledge base handles storage, organization, and reasoning — with the two connected through a memory system.

Three Major Technical Challenges

To actually deploy such an agent architecture, the paper identifies several unavoidable technical hurdles.

Connecting Perceptual Data to Structured Knowledge

The first challenge is how to establish a reliable link between sensor data and the structured data stored in memory. Raw perceptual signals from the real world are continuous, noisy, and unstructured, while an agent's knowledge base must exist in structured form to be usable for reasoning and planning. Reliably mapping the former to the latter is the critical bridge between perception and cognition.

Goal-Directed Decision-Making and Planning

The second challenge revolves around decision-making. Agents need to make decisions around their own goals and plan how to achieve them. This requires not only immediate judgment, but also the ability to reason over future sequences of actions — precisely the strength of symbolic planning methods, and a part where pure end-to-end models often fall short.

The most representative symbolic planning methods are Automated Planning techniques, with canonical implementations including formal languages like STRIPS and PDDL, as well as search-based planners. These approaches express world states, action preconditions, and effects as logical predicates, generating plans by searching for action sequences from an initial state to a goal state. Their advantages lie in strong interpretability, guaranteed logical consistency of plans, and support for multi-step lookahead. By contrast, end-to-end deep reinforcement learning models embed the perception-to-action mapping implicitly in network weights, making it difficult to extract verifiable reasoning chains — a significant limitation in scenarios requiring long-horizon planning or goal interpretability. The emerging field of Neuro-Symbolic Planning attempts to use neural networks to learn action models while retaining the reasoning structure of symbolic planners, aiming to capture the best of both worlds.

Coordinating Individual and Collective Intelligence

The third challenge involves multi-agent collaboration. The paper emphasizes that combining individual intelligence with collective intelligence requires coordination across multiple agents. This is also the core issue pointed to by the article's subtitle, "From Cognition to Collective Intelligence": the capabilities of individual agents must ultimately be amplified within a collaborative network.

Redefining "Trustworthiness": From Behavior to Cognition

One of the paper's most thought-provoking contributions is its redefinition of trustworthiness in autonomous systems.

The authors argue that, unlike traditional systems, an agent's trustworthiness cannot be limited to behavioral attributes alone. It also encompasses a crucial cognitive dimension — one whose validity depends on how the agent uses its knowledge when making decisions.

In other words, evaluating whether an autonomous system is trustworthy means looking not just at "what it did," but at "what knowledge it based its judgment on, and how." An agent whose surface behavior appears correct but whose internal reasoning logic is flawed remains questionable in terms of trustworthiness. Based on this insight, the paper proposes several directions for developing agent trustworthiness evaluation methods, extending the assessment from pure output verification to an examination of the cognitive process itself.

In traditional systems engineering, trustworthiness is typically measured through behavioral metrics such as reliability, safety, and availability, relying on formal verification, test coverage, and failure mode analysis. However, when a system contains learning components, this approach faces fundamental challenges: the behavioral space of neural networks is vast and difficult to exhaustively test, and models may produce unpredictable outputs outside their training distribution. Extending trustworthiness to the cognitive dimension means the evaluation framework must answer questions like "why did the agent make this decision" and "is its knowledge base reliable, and does it contain biases?" This aligns closely with the AI field's recent research directions in Explainability, Robustness, and Value Alignment, and echoes the policy trend of regulatory frameworks like the EU AI Act requiring auditability for high-risk AI systems.

The Gap Between Vision and Reality

The paper does not rest on an optimistic portrayal of its framework, but closes with a critical stance. The authors candidly acknowledge that a substantial gap remains between the grand vision of autonomous multi-agent systems and the current state of the technology.

This honest assessment deserves emphasis. While current AI is advancing rapidly on individual capabilities like perception and generation, building an autonomous agent system with persistent memory, reliable planning, collaborative capability, and trustworthy, evaluable behavior still faces a large number of unresolved problems. Genuinely integrating connectionist perceptual ability, symbolic reasoning capability, and the rigor of systems engineering into a unified whole is a threshold that must be crossed on this path.

Summary

The value of this paper lies in its refusal to remain within a single technical track, instead examining autonomous systems within the broader landscape of AI development: a long-term memory-centric agent architecture, the connection between perception and knowledge, goal-directed decision-making and planning, multi-agent coordination, and trustworthiness evaluation that encompasses the cognitive dimension — together, these form a blueprint for designing and evaluating autonomous systems. For researchers and engineers focused on next-generation AI systems engineering, it offers a reference framework that