Project Lumina: Building a Brain-Inspired AGI from Scratch and the T4 VRAM Lesson

In the AI field, most teams choose to fine-tune existing frameworks, but one group has decided to start from scratch and replicate the human brain using biological mechanisms. The founder and CEO of AGI Systems Directorate recently unveiled Project Lumina, an autonomous AI agent and large language model built entirely from the ground up. The project is currently in the pre-training phase, targeting a parameter scale of over 100 billion, with an official release planned for next month.

Why Mimic Biological Mechanisms?

Most current large models are based on the Transformer architecture and rely on static attention matrices. The Project Lumina team believes that to truly move toward artificial general intelligence, we cannot keep copying existing technology templates.

They look for answers in how the human brain works. Project Lumina integrates 500 to 1,000 different biological mechanisms at its foundation, each carrying out an independent function. These mechanisms are not simply stacked together but are distinguished by anatomical and functional structure, including:

• Free Energy Principle
• Synaptic plasticity and dynamic weight adaptation
• Frontal lobe replication circuits
• Fluid intelligence benchmarks
• Dorsolateral prefrontal cortex (dLPFC)
• Ventromedial prefrontal cortex (vmPFC)
• Hippocampal-entorhinal complex

According to the founder, the goal of these mechanisms is to simulate how biological systems process information, adapt in real time, and maintain persistent states without causing an explosion in computational complexity.

The Free Energy Principle, proposed by neuroscientist Karl Friston, is a unified theory in computational neuroscience that explains how the brain works. Its core idea is that biological systems maintain their stability by minimizing "free energy" (i.e., the discrepancy between predictions and actual sensory input). The brain continuously generates predictive models of the world; when predictions do not match input, it either updates the model (learning) or changes behavior to match the prediction (action). In AI, this principle can be formalized as variational inference, enabling agents to perform active reasoning and minimize uncertainty, and it is regarded as one of the important theoretical paths toward autonomous intelligence.

Synaptic plasticity refers to the property by which the strength of connections between neurons changes with activity history, and it is the basis of biological learning and memory. Common forms include long-term potentiation (LTP) and long-term depression (LTD). In artificial neural networks, traditional training uses global backpropagation to update weights all at once, whereas biological systems rely on local, online, sparse update rules. Project Lumina attempts to introduce dynamic weight adaptation mechanisms into the model so that the network can continuously adjust connection strengths even during inference, thereby gaining continual learning and environmental adaptation capabilities.

The dorsolateral prefrontal cortex (dLPFC) and the ventromedial prefrontal cortex (vmPFC) are two key subregions of the prefrontal cortex. The dLPFC is mainly involved in working memory, rule learning, and cognitive control and is the core of "cold cognition"; the vmPFC is associated with value judgment, emotion regulation, and risk-benefit assessment in decision-making and belongs to the "hot cognition" system. Together they support flexible goal-directed behavior. In a brain-inspired AI architecture, simulating these two regions helps separate "reasoning control" from "value assessment" functions, enabling the agent to both plan and weigh trade-offs in complex environments.

Project Status: Pre-training in Progress

The founder readily acknowledges that Project Lumina is not yet complete. Pre-training has been running for about two days, and the team is at a critical stage. He chose to share progress in the form of a conversation rather than a document and promised to personally reply to questions in the comments.

This openness is rare in AI R&D, but it also means the project's final performance remains to be verified.

The Painful Lesson of T4 GPUs and the Reality of Compute

During the project's advancement, the team stumbled on hardware. The founder initially insisted on trying to run the full training pipeline on ordinary T4 GPUs, attempting CPU offloading, aggressive memory garbage collection, gradient checkpointing, custom chunking, and various other methods.

Each time, they were defeated by out-of-memory errors. Eventually, the team abandoned this fixation and moved to the standard workflows of H100, H200, and TPU.

The founder's summary is quite blunt: in the current AI race, compute decides victory. With massive compute, even mediocre ideas can win by piling on parameters; with breakthrough ideas but no compute, you will only hit a wall.

This bottleneck is exactly what the team aims to solve next—lowering the compute barrier for independent developers and researchers.

Gradient checkpointing is a technique that trades computation for VRAM: instead of storing all intermediate activations during the forward pass, it recomputes some activations during backpropagation, thereby reducing memory usage from being proportional to the number of layers to being proportional to the checkpoint interval. CPU offloading moves some model parameters or optimizer states to CPU memory and transfers them back to the GPU when needed to relieve VRAM pressure, but it introduces significant data transfer overhead. Custom chunking usually refers to splitting long sequences or large tensors into small chunks for batch processing to fit limited VRAM. These methods are especially common on inference cards like the T4 with only 16GB of VRAM, but for pre-training at the hundred-billion-parameter level, their benefits are far from enough to make up for the fundamental capacity and bandwidth gap in hardware, which is also why the team ultimately moved to H100/H200/TPU.

Next Steps: Tens of Billions of Parameters and Quantization Optimization

Project Lumina plans to officially release next month, with a parameter scale exceeding 100 billion, while focusing on optimizing quantization technology so that it can run efficiently on basic hardware.

The founder used a vivid comparison: the human brain operates on only 12 to 20 watts of power, while data centers consume resources comparable to an entire river. Closing this gap is Project Lumina's long-term goal.

On the eve of release, the team is soliciting community opinions: which platform should the first public version be deployed to in order to obtain the most valuable technical feedback? This question is also left to developers following the project.

From T4 VRAM errors to a hundred-billion-parameter brain-inspired architecture, Project Lumina's path is full of ambition and also full of uncertainty. Whether it can deliver on its promises, next month's release will provide the first answer.

Frequently Asked Questions

What is Project Lumina?

Project Lumina is an autonomous AI agent and large language model built from scratch by AGI Systems Directorate, integrating multiple biological mechanisms with the goal of simulating how the human brain processes information.

Why abandon T4 GPUs?

The team tried CPU offloading, gradient checkpointing, and various other memory optimization methods, but T4 GPUs still could not handle the full training pipeline, so they ultimately moved to H100, H200, and TPU.

When will Project Lumina be released?

The team plans to release the first public version next month, with an expected parameter scale of over 100 billion, optimized for quantization to adapt to basic hardware.

One-line Summary

AGI Systems Directorate unveils the brain-inspired AI project Project Lumina, with a hundred-billion-parameter version coming next month.

Paragraph Summary

The founder and CEO of AGI Systems Directorate unveiled Project Lumina, an autonomous AI agent and large language model built from scratch. The project integrates 500 to 1,000 biological mechanisms, including the Free Energy Principle, synaptic plasticity, and prefrontal cortex circuits, in an attempt to simulate how the human brain processes information. Pre-training has been running for about two days, and after repeatedly encountering out-of-memory errors on T4 GPUs, the team moved to H100, H200, and TPU. The project plans to release a hundred-billion-parameter version next month and focus on quantization optimization to adapt to basic hardware. The founder solicits community opinions through open dialogue and emphasizes the decisive role of compute in the AI race.