Making Rocks Think: A Philosophical Exploration of Generative AI and Information Compression

A viral Reddit post, summed up by the comment "I knew it was theoretically possible, but never thought I'd live to see it," captures the complex feelings many people have about today's AI technology. The comment section spiraled from jokes about "putting electricity in rocks to make them think" into a surprisingly deep discussion about the nature of information, possibility spaces, and the principles behind generative AI. What seemed like wild speculation actually touched on one of the most important lenses for understanding modern AI.

The Technical Reality Behind "Making Rocks Think"

The most incisive comment in the thread was: "We put lighting in rocks to make them think." It's a joke, but it's also an accurate description of the physical reality of computers and AI — silicon chips are, at their core, precisely engineered "rocks" (sand, silicon), and through combinations of electrical switching, they make the leap from simple logic operations to complex neural network inference.

Another commenter offered a philosophical counterpoint: "Our brains are just randomly evolved meat computers. If some organic chemicals can randomly organize into self-replicating machines, then of course we can make rocks think." The analogy isn't rigorous, but it points to an intriguing truth — intelligence may not be the exclusive domain of carbon-based life, but rather an emergent property of certain information-processing structures.

"Every Possible Video Already Exists" — A Library of Babel Thought Experiment

The most imaginative thread in the discussion came from a user's sudden realization: that "every hypothetical event has a corresponding video — we just need to know the right bit sequence to watch it." Mathematically speaking, all possible combinations of bits already "exist," so that video "exists" too — we just don't know how to reconstruct the data.

This isn't pure fantasy. Someone immediately linked libraryofbabel.info — a website based on Borges' The Library of Babel that algorithmically "contains" every possible combination of text. Any book ever written or yet to be written can theoretically be found within it. Extending this logic to images and video leads to the conclusion that "all possible media already exists somewhere in the space of possibilities."

This line of thinking reveals a core insight: the existence of information and the accessibility of information are two very different things. The possibility space is astronomically vast, and the real challenge is locating that one meaningful bit sequence.

Borges wrote The Library of Babel in 1941, describing a universe-spanning library containing every possible book — each 410 pages, 40 lines per page, 80 characters per line, drawn from a finite set of 25 symbols. This means the library "contains" everything ever written or yet to be written, along with endless meaningless gibberish. libraryofbabel.info digitizes this thought experiment: it doesn't actually store all the text, but uses a deterministic algorithm to map any coordinate to a character sequence, and vice versa. This makes it possible to "search" for any text — type in a passage and the site tells you its "location" in the library. The design reveals a subtle point: the real challenge was never whether the content exists, but that the index itself requires as much information as the content. For video, one minute of HD footage requires several gigabytes of data, and the number of possible arrangements far exceeds the number of atoms in the observable universe — exhaustive search is physically impossible.

The Deep Connection Between Compression and Generative AI

The most technically valuable contribution came near the end of the thread, when a commenter wrote: "That's not quite how it works, but your intuition is pointing in the right direction. Look into the similarities between compression and generative AI, semantic compression, and the Hutter Prize."

This comment points to the key thread for understanding modern AI. There is a profound connection between compression and intelligence: a system that can efficiently compress data must have "understood" the patterns and structure within that data. The Hutter Prize — a competition rewarding the best lossless compression of Wikipedia text — is built precisely on the assumption that better compression equals better understanding.

Generative AI as "Lossy Semantic Compression"

Large language models and image generation models are, at their core, compressing enormous amounts of training data into a finite set of model parameters. GPT-style models use billions of parameters to "memorize" the statistical patterns of the entire corpus of internet text; diffusion models learn how to reconstruct images consistent with human visual experience from noise.

This echoes the earlier thought experiment about "all videos already existing": generative AI doesn't blindly search through infinite possibility space — it learns the distribution of the "meaningful regions" within that space. What it captures isn't complete bit sequences, but the "compressed rules" for generating plausible content. This is why AI can generate images it has never seen before — it performs interpolation and sampling within a compressed semantic space.

The Hutter Prize was established by AI researcher Marcus Hutter in 2006, rewarding algorithms that compress 100MB of English Wikipedia text to the smallest possible size, with prize money tied to compression ratio. Its theoretical basis comes from Kolmogorov Complexity: the "true length" of a piece of data equals the length of the shortest program that can generate it. To efficiently compress natural language, a compressor must learn grammar, semantics, common sense, and world knowledge — so compression ability directly measures the depth of "understanding" of the data's patterns. Shannon's foundational 1948 paper on information theory already proved that the optimal code length equals the information entropy, meaning the compression limit is determined by the statistical structure of the data. The training objective of large language models (predicting the probability of the next token) is mathematically equivalent to minimizing cross-entropy — that is, maximizing compression rate. This through-line from Shannon's information theory to modern LLMs explains why "compression equals understanding" is not merely a metaphor, but a proposition with rigorous mathematical foundations.

From Armchair Speculation to Serious Insight

What's worth noting is that the original poster received considerable pushback, but the commenter who gave the research directions remarked: "Not sure why you're getting so much pushback — your intuition is pointing in the right direction."

This reflects a common phenomenon: many intuitions about the nature of AI, though imprecisely stated, often touch on real technical principles. The bridges between information theory, compression theory, and machine learning have long been serious areas of academic research. From Shannon's information theory to Kolmogorov Complexity to today's generative models, the idea that "compression equals understanding" runs as a consistent thread throughout.

For readers who want to explore this further, follow the keywords the commenter provided: semantic compression, the relationship between compression and generative models, and the Hutter Prize. These concepts will help you understand, from a more fundamental perspective, why "making rocks think" is no longer science fiction — it's the technological reality of our time.

Conclusion

A casual Reddit post unexpectedly connected the physical reality of silicon-based computing, mathematical speculation about possibility spaces, and the information-theoretic principles of generative AI. These discussions remind us that understanding AI requires not just an engineering perspective, but also philosophical and information-theoretic thinking. When we marvel at AI's seemingly ex nihilo creativity, perhaps what we should recognize is this: it has learned to navigate the vast expanse of possibility space and find the path toward meaningful information.