Triad Architecture for AI's 'Language of Thought': Noun-Verb-Conjunction

In today's world dominated by large language models, a new idea about the internal representations of neural networks is sparking discussion on Reddit. The idea proposes using "noun-verb-conjunction" triads as AI's "language of thought," structuring simple propositions through attention heads to form interpretable, scalable reasoning units.

Core Idea: Expressing Thoughts with Triads

According to the original Reddit post, the scheme suggests that neural networks contain multiple layers, at least one of which consists of multiple attention heads. Each attention head holds a triad: a noun, a verb, and a conjunction. For example, "The cat ate the mouse and then died" can be broken down into basic units like "(empty) cat ate," "also mouse was eaten," "and cat is a Siamese cat," "then cat died," each with a conjunction to express logical relations.

Here "also" is given a special meaning: indicating that two events are different aspects of the same event—a usage that doesn't exist in English, but the author believes it can serve as a logical glue for the internal language. In this way, the network can decompose complex sentences into multiple simple triads and reassemble them as needed.

Attention Head is the basic computational unit in the Transformer architecture. Each attention head captures dependencies between different positions in the input sequence by computing interactions between Query, Key, and Value, thereby dynamically aggregating information. In a standard Transformer, multi-head attention runs multiple attention heads in parallel, each learning to focus on different patterns or relationships. This idea proposes having each attention head "hold" a triad, effectively assigning each head an explicit symbolic role rather than the traditional distributed representation. This design aims to transform the implicit, hard-to-interpret attention patterns in neural networks into explicit, structured logical units.

Compositional Generalization refers to a model's ability to understand and generate novel combinations not seen during training. For example, if a model learns "cat chases mouse" and "dog chases ball," it should understand the new combination "cat chases ball." Traditional neural networks perform poorly on compositional generalization, often relying on surface statistical patterns. Structured triad representations, by decomposing complex propositions into basic units (noun-verb-conjunction) and explicitly combining them, are expected to enhance the model's sensitivity to compositional structure, thereby generalizing better to new combinations. This aligns closely with one of the core goals of neuro-symbolic AI—combining the perceptual capabilities of neural networks with the reasoning capabilities of symbolic systems.

Extended Markers: Tense, Modality, and Questions

Beyond basic triads, each attention head can attach more markers to enrich expressiveness. The original post lists several optional markers: articles (the, a, same), tense (past, present, future), modal verbs (could, would, should, might), "try to," and question marks. For example, with a modal marker, "therefore the mouse might have been poisoned" can be expressed as "therefore the-mouse might-have-been-poisoned."

These markers allow triads to describe not only facts but also possibilities, intentions, and questions. The author emphasizes that the goal is to give neural networks a simple, structured internal language for forming and combining thoughts—a "language of thought."

Handling Graph Structures: Chunking and Progressive Understanding

The original post further points out that in principle an entire book can be represented as a huge interconnected triad graph. But neural networks don't need to process the whole graph at once, just as humans don't understand an entire book in one go. Humans divide a book into manageable units (like sentences), understand these units, and gradually build an overall understanding of the story.

The proposed scheme is similar: group triads for processing, with the network carrying relevant information and generating new triads. This progressive processing avoids the complexity of handling the entire graph at once while retaining the structural advantages of symbolic reasoning.

Neuro-Symbolic AI is a subfield of artificial intelligence that aims to combine the powerful learning capabilities of neural networks with the interpretable reasoning capabilities of symbolic systems. Symbolic systems (such as logic rules and knowledge graphs) excel at handling structured knowledge and precise reasoning but struggle to learn from raw data; neural networks excel at pattern recognition and representation learning but lack interpretability. Neuro-symbolic AI attempts to bridge this gap, for example by embedding symbolic representations into neural networks or using neural networks to guide symbolic reasoning. The triad architecture proposed in this article falls precisely into this category: it attempts to establish a symbolic "language of thought" within neural networks, making internal representations both flexible like neural networks and readable and structured like symbolic systems.

Potential Advantages and Open Questions

The appeal of this triad architecture lies in its interpretability: each triad corresponds to a simple proposition, and the network's internal state can be mapped to human-readable symbols. Additionally, structured representations may enhance compositional generalization, making it easier for models to handle new combinations.

However, the idea is still conceptual and lacks experimental validation. How to automatically extract triads from raw text, how to handle complex relationships between triads (such as coreference and ellipsis), and how to integrate with existing neural network training paradigms are all questions that need exploration.

The original poster also acknowledges that this is merely a proposal intended to stimulate discussion. For developers concerned with AI interpretability and neuro-symbolic integration, this approach may offer new inspiration.

Conclusion

The "noun-verb-conjunction" triad offers a lightweight symbolic representation scheme, attempting to establish a structured language of thought within neural networks. Although not yet empirically tested, it touches on a core question in AI: how to make machines' internal representations closer to human-understandable logical forms. With the revival of neuro-symbolic AI, such explorations deserve continued attention.

One-line Summary

Reddit idea: using noun-verb-conjunction triads as AI's language of thought for interpretable reasoning.

Paragraph Summary

This article introduces a new idea discussed on Reddit: using "noun-verb-conjunction" triads as the internal language of thought for neural networks. Each attention head holds a triad, which can be augmented with markers for tense, modality, and questions, thereby decomposing complex sentences into simple propositions and reassembling them. Entire knowledge can be represented as a triad graph, and the network progressively builds understanding through chunked processing. The scheme emphasizes interpretability and compositional generalization, but is still conceptual, lacks experimental validation, and needs to address challenges such as automatic triad extraction and relationship handling.