From Rocks to Rules: A Semantic Ladder for AI Alignment

A grand theoretical framework about language, norms, and AI value alignment is stirring discussion in the LessWrong community. Starting from the question of "how two reasoners with identical predictions but different models can talk to each other," the author constructs a semantic ladder that climbs from nouns and verbs all the way up to rules and pacts, and attempts to use it to answer a core problem in alignment: under what conditions is your "murder" a function of my "murder."

How Two Reasoners Communicate: Starting from Interoperable Semantics

The starting point of this framework is Wentworth's "interoperable semantics": imagine two Bayesian reasoners who have identical predictions about the world but potentially completely different internal world models. They exchange messages and try to communicate, and the symbols composing those messages can be arranged in layers by "binding-types," forming a ladder.

Each rung of the ladder is defined by "which lower-level types must be mutually understood before it can be translated." The n-th level is specified by translatability conditions on the (n-1)-th level. The author calls this the "Wentworth native framework"—it promises that two Solomonoff inductors sharing no common language can still learn to talk about drinks at a bar, even though the original account stops short at "how to talk about kinds of drinks."

The author sets a strict acceptance criterion for himself: if a concept cannot be cashed out in terms of everyday objects and situations, it isn't worth spending glucose thinking about. "The blue cup is on the table, and I pick it up"—any semantic model that equivocates on this sentence is merely sophistry.

Solomonoff inductor refers to a universal predictor built around Solomonoff inductive inference: it assigns prior weights to all computable programs based on program length (typically Kolmogorov complexity or a prefix-code measure), then produces a weighted mixture prediction over observed data. Since any computable hypothesis lies within its search space, a Solomonoff inductor theoretically possesses extremely high inductive capacity and serves as the foundation for general intelligence theories like AIXI. Two Solomonoff inductors sharing the same training observations will produce entirely identical probabilistic predictions about the world, yet the internal programs each uses to realize those predictions may bear no resemblance to each other—precisely why the author needs to introduce an additional interoperable semantic layer to characterize how they move from "predicting the same thing" to "being able to talk about the same thing."

Gärdenfors's conceptual spaces refers to the "conceptual spaces" theory proposed by cognitive scientist Peter Gärdenfors: concepts are represented as regions or points in a geometric space spanned by several quality dimensions (such as hue, brightness, pitch, temperature). Objects are embedded in this space according to their values along each dimension, and concepts correspond to convex regions within it. The author uses this to understand nouns as "clusters in object space that words point to" and adjectives as coordinates within such clusters, reducing lexical semantics to geometric structure so that the lower rungs of the ladder can rest on relatively rigorous mathematical objects.

The Lower Half of the Ladder: Nouns, Verbs, and the Smear Phenomenon of Modifiers

The lower half of the ladder is built on Gärdenfors's conceptual spaces and the picture of "words pointing to clusters in object space." Clusters in object space give nouns; coordinates within clusters give adjectives ("hard," "blue," "delicious"). The author then adds several key operations:

• predicativize: casting internal coordinates (modifiers) into unary predicates—"hard" becomes "is hard"
• saturate (in the Fregean sense): filling the slots of a function to form a complete sentence
• arity-climb: extending from unary predicates to prepositions and binary/n-ary relational verbs
• temporalize: introducing time, enriching configuration space into path space, giving verbs semantics of motion, transfer, transformation, and so on

This framework makes a falsifiable prediction: wherever the ladder sets up a bridge, cross-linguistic typology should exhibit "smear"—an unstable word class that is neither here nor there, stuck between two levels. Korean "descriptive verbs" and Lojban's brivla sit precisely on the first bridge, while the opposition between locative and path prepositions (and Talmy's verb-framed/satellite-framed typology) sits on the second.

Saturate in the Fregean sense comes from Frege's function-argument analysis: an unsaturated expression (such as the predicate "is hard" or the relational word "gives") carries slots that must be filled by arguments, and only after saturation can it form a complete sentence with a truth value. The author borrows this operation to "fill in" incomplete meanings like adjectives or relational words into propositions, thereby linking modifiers, predicates, and sentences within the ladder.

Talmy's verb-framed/satellite-framed typology was proposed by cognitive linguist Leonard Talmy to distinguish how languages encode motion events: verb-framed languages (such as Spanish and French) place path information in the main verb ("entrar" meaning to go in), while satellite-framed languages (such as English and German) place path in "satellite" elements peripheral to the verb (such as "go in," "walk out"). This opposition is often thought to characterize the unstable zone on the second bridge of the ladder, and is also the region where smear phenomena are expected to concentrate.

Lojban's brivla refers to the predicate word class in Lojban: each brivla comes with several argument slots (its "place structure") that must be filled in fixed positions to form a complete proposition, and the same word can slide between verb, adjective, and even noun functions depending on how it is filled. It therefore resembles both a verb and an adjective—precisely the kind of unstable word class the author describes as stuck between two levels, and one of the ready-made corpus pieces of evidence for the theory.

The Upper Half: The Two Levels of Rules and Pacts

The real innovation lies in the upper half of the ladder. Starting from odd adverbs like "never" and "always" and odd modal verbs like "should," the author proposes a new operation—constrain: a constraint screened over an entire set of trajectories is a rule. A rule is not a property check on individual events—precisely why it needs its own level.

A new operation—deontic-lift—turns "is P" into "let everything be P," leaping from description to prescription. The author invokes Hume's guillotine to point out that no matter how high you stack descriptions, you cannot build an "ought." And the smear phenomenon appears again—generics and the gnomic mood are precisely the bridge climbing from adverbs to rules; "dogs bark," "the pope is Catholic," and "lemons are yellow" appear to be descriptions but are in fact the politics of redrawing boundaries.

Further up, rules climb through "convention"—the messiest smear of all (custom, norm, tradition, usage, and similar word classes are extremely unstable)—to the final level: pacts. The author offers a characterization: let R be the space of expressible rules, S=2^R the set of all rule sets, and A={A_i} a set of agents. A pact is an element of S^n, in which all agents agree to abide by their own s_i on the condition that others abide by their respective s_{-i}.

Pact space has real structure: the joint satisfiability of rule sets is downward closed, forming precisely an abstract simplicial complex K(R). Two pacts are compatible if and only if, for every agent who is a party to both, their two rule cells share a co-cell. Here Lindenbaum's lemma quietly appears—consistent rule sets can always be extended to complete consistent rule sets.

Hume's guillotine (also known as the is-ought problem) was proposed by Hume in A Treatise of Human Nature: from descriptive premises alone (statements about "is") one cannot logically derive normative conclusions (about "ought"). The author uses this to show that no matter how high the descriptive ladder is stacked, the deontic-lift step is a genuine leap, and to explain why value questions cannot be reduced to translation at the noun level.

Lindenbaum's lemma is a classic result in mathematical logic: any consistent set of formulas can be extended to a maximal consistent set (thereby yielding a complete theory in a countable language). The author uses it here to illustrate the geometric properties of pact space—the downward-closed joint satisfiability of rule sets forms an abstract simplicial complex K(R), and the fact that consistent rule sets can always be extended to complete consistent sets corresponds to the fact that cells in the complex can always be contained within some maximal cell.

An abstract simplicial complex is a combinatorial abstraction in topology of a finite geometric simplicial complex: it consists of a vertex set and a collection of subsets (faces or cells) satisfying a "downward closure" condition—if a subset is in the complex, then all its subsets are too. Only when the joint satisfiability of rules satisfies the same downward closure does the entire pact space possess a complex structure; this allows pact compatibility to be expressed as two cells intersecting on some common co-cell, thereby bringing mature combinatorial-topological tools to bear on the overall consistency of normative systems.

AI Alignment Reflections Sparked by the Ladder

Why is this semantic philosophy relevant to AI alignment? The author offers four deep implications:

1. Value learning and value loading: In domains where there are real stakes, values are entirely rules and pacts—"murder must not be crossed" is a constraint on a trajectory set, and disputes over "coercion" and "consent" are all third-level politics. Treating alignment as a noun-level translation problem is "building a basement and calling it a tower." What is currently missing is precisely a "translatability theorem for rules": natural latent variables can tell you when your "dog" is a function of my "dog," but no one has given the corresponding theorem for "murder" (a constraint on a trajectory set).
2. Diagnosing ontological mismatch: When a powerful optimizer's ontology diverges from yours, "which level of the ladder" is the first diagnostic question.
3. The puzzle of sample efficiency: Children learn vocabulary from extremely little exposure (fast mapping, syntactic bootstrapping) because new words arrive with pre-types, directly cutting the hypothesis space by several orders of magnitude.
4. Embedded agency: "I," "here," and "now" are coordinates of an agent in its world model, and all value references that must be delivered to a system carry indexicals that must be bound within the extension of being bound.

The author candidly admits this is a tower deliberately built half in public: scaffolding exposed, mortar still wet, holes labeled with his own name. The open problems he offers bounties for include: the mathematics underpinning the whole framework, natural rule conditions, where discrete clusters come from, adopt-irreducibility, and translatability under ⌜·⌝. This theory accepts everything and predicts nothing—only a tower that can be broken is worth building.

One-line Summary

A semantic ladder from nouns to pacts offers a leveled diagnostic framework for AI value alignment.

Paragraph Summary

This article proposes a "semantic ladder" framework that climbs from nouns and verbs all the way up to rules and pacts, starting from Wentworth's interoperable semantics: how two Bayesian reasoners with identical predictions can communicate despite different models. The lower half of the ladder draws on Gärdenfors's conceptual spaces, reducing nouns to clusters in object space and adjectives to coordinates within clusters, and builds upward through operations such as predicativize, saturate, arity-climb, and temporalize. The upper half proposes constrain and deontic-lift, understanding rules as constraints on trajectory sets and pacts as compatible cells in a rule-set space, and predicts that the cross-linguistic "smear" phenomenon (Korean descriptive verbs, Lojban brivla, the gnomic mood, etc.) will appear at every bridge. From this the author derives four AI alignment implications: value learning is fundamentally a rule- and pact-level problem, ontological mismatch can be located at specific ladder levels, sample efficiency stems from the pre-typing of vocabulary, and embedded agency is realized through the placement of indexicals.