Using Code Golf and RLHF to Train AI to Explore the Demoscene

When AI Meets the Demoscene: An Experiment in Code Aesthetics and Incompleteness

At a time when AI-generated content is becoming increasingly homogeneous, a developer named datavorous shared a rather unconventional experiment on Reddit: fine-tuning on "golfed" JavaScript code and using self-rated RLHF (reinforcement learning from human feedback) on 40,000 samples to train a model to explore the syntax of the Demoscene and the possibilities of incomplete code. What makes this experiment unique is that it does not chase improvements in general capabilities, but instead turns its gaze toward a niche programming culture centered on extreme compression and visual aesthetics.

Why the Demoscene?

The Demoscene is a computer subculture that originated in the 1980s, whose participants take pride in creating stunning audiovisual effects within extremely small program sizes. This culture naturally emphasizes the "syntax" of code—not syntax in the sense of language specifications, but how to organize logic, call hardware, and compress data under byte constraints to ultimately achieve aesthetic expression. The "demoscene syntax" mentioned in datavorous's post title is precisely an abstraction of this creative paradigm.

Introducing this syntax into model training means the model must not only learn to write code, but also learn to engage in creative expression under constraints. This forms an interesting contrast with the goals of current mainstream code generation models, which pursue functional correctness and readability.

Code Golf is a form of programming competition in which a specified function is implemented using as few characters as possible. Participants must complete algorithms under strict character limits, often using language features, implicit conversions, and concise syntax to compress code. The resulting output is usually highly compact and poorly readable, but extremely dense in logic. Code golf and the Demoscene share the spirit of "creating under constraints," but the former focuses on minimizing character count, while the latter pays more attention to achieving audiovisual effects within an extremely small binary size. Using golfed JavaScript as training data is precisely intended to let the model learn this extremely compressed coding style.

Training Strategy: From Golfed JS to Self-Rated RLHF

According to datavorous's description, training is divided into two key stages. The first stage is fine-tuning on "golfed JS." Code golf is a form of programming competition in which functionality is implemented with the shortest possible character count; its output is often highly compressed and extremely unreadable, yet extremely dense in logic. Fine-tuning a model on this kind of data is equivalent to exposing it to an extremely stylized code distribution, enabling it to more acutely capture the conciseness and cleverness prized by the Demoscene.

The second stage is RLHF. Interestingly (this is to emphasize the experimental design, not as a cliché), datavorous did not rely on external human annotation, but instead had the model self-rate the aesthetics of about 40,000 samples, and then performed reinforcement learning based on these ratings. This "self-rated RLHF" approach provides a feasible path under limited resources, but it also introduces the risk of the model reinforcing its own biases.

RLHF (reinforcement learning from human feedback) is a method that trains a reward model on human preference data and then fine-tunes a language model with reinforcement learning. The standard process includes: collecting human comparisons or ratings of model outputs, training a reward model to predict human preferences, and then using reinforcement learning algorithms (such as PPO) to optimize the model so as to maximize reward. The "self-rated RLHF" in this experiment replaces human annotation with the model's own ratings, reducing costs, but the reward signal comes from the model itself, which may amplify existing biases and lead to a self-reinforcing loop.

The Possibilities of Incomplete Code

Another intriguing point is "the possibilities of incomplete code" in the title. In traditional software engineering, incomplete code is usually regarded as a defect or a half-finished product. But in the context of the Demoscene, incompleteness may mean an open-ended creative starting point—a piece of code with blank space can stimulate the viewer's imagination and may also become the seed for subsequent iterations.

datavorous's experiment attempts to make the model understand and generate this kind of "incompleteness," rather than single-mindedly pursuing complete, runnable output. This challenges our usual criteria for evaluating the quality of AI code generation.

"Incompleteness" in the Demoscene often manifests as deliberate blank space, fragmented effects, or open endings, in order to stimulate the viewer's imagination or leave room for subsequent iterations. In the context of code generation, generating incomplete code means the model needs to understand the aesthetic value of "partial implementation," rather than merely pursuing runnability. This requires training data to contain a large number of non-complete fragments and the design of evaluation metrics that can reward openness and inspirability, rather than traditional correctness measures.

The Significance and Limitations of the Experiment

The value of this experiment lies in exploring an alternative possibility for model training objectives: not pursuing scores on general benchmarks, but deeply customizing around the aesthetics and syntax of a specific subculture. It echoes the open-source community's expectations for "small and beautiful" and "stylized" AI tools.

However, because the original information comes only from the Reddit post title and a brief description, we cannot know details such as model size, training data volume, or specific evaluation methods. The effectiveness of self-rated RLHF also remains to be verified—when a model scores itself, it can easily fall into a loop of "talking to itself." In addition, the Demoscene community's attitude toward AI-generated content is still unclear, and whether such experiments can be accepted by the community remains unknown.

Vertical culture customized training refers to data selection and objective design aimed at the syntax, aesthetics, and constraints of a specific subculture or art form, rather than pursuing general capabilities. Compared with general large models, this type of method has a smaller data scale and a more focused objective, and can capture the unique style of niche fields, but its generalization ability is limited, and it depends on a deep understanding of the target culture. Code golf, the Demoscene, and poetry generation can all serve as testing grounds; the key lies in building an evaluation system that can reflect the values of that culture.

Implications for AI Creators

For those interested in AI programming and creative tools, datavorous's attempt offers an idea: rather than competing head-on with large model vendors on general capabilities, it may be better to go deep into a vertical culture or art form and use customized training strategies to capture its unique syntax. Code golf, the Demoscene, and even poetry generation may all become testing grounds for this kind of "syntax-driven" training.

At the same time, the perspective of "incomplete code" also reminds us that the evaluation system for AI-generated content can be more diverse. Completeness and correctness are certainly important, but openness and inspirability are equally worth including in consideration.

(Note: This article is based on information from a public Reddit post and does not include screenshots. Experimental details are subject to the original author's subsequent disclosures.)