Genetic Algorithms + Neural Networks: How a 3D Robotic Arm Evolves to Reach Its Target

A Fascinating Evolution Experiment

Recently, a developer shared his experimental results on Reddit: by combining a Genetic Algorithm (GA) with a Multi-Layer Perceptron (MLP) neural network, he enabled a population of 3D robotic arms to autonomously "evolve" the ability to reach a target. The entire project was completed through collaboration with ChatGPT and Codex (what the author calls "vibecoding"). Despite some bumps along the way, the system ultimately achieved rapid convergence.

This seemingly niche project actually encapsulates the core ideas of evolutionary computation and neural network control, making it well worth a deeper look.

Technical Breakdown: How Genetic Algorithms and Neural Networks Work Together

Genetic Algorithms: An Optimization Engine Inspired by Natural Selection

Genetic algorithms are a class of optimization methods inspired by Darwinian evolution, first proposed by John Holland at the University of Michigan in the 1960s and later popularized by David Goldberg and others in the 1980s. The core idea is to encode candidate solutions as "chromosomes" and search for optimal solutions by simulating the process of natural selection.

In this example, the genetic algorithm maintains a "population" — a batch of robotic arm controllers with varying parameters. In each generation, the system evaluates how well each individual reaches the target (the fitness function), retains top performers, and produces the next generation through "crossover" and "mutation." Specifically, the selection operator determines which individuals qualify as "parents" (commonly using tournament selection or roulette wheel selection); the crossover operator mixes parameters from two parents according to some rule (such as single-point crossover, uniform crossover, or arithmetic crossover); and the mutation operator applies random perturbations to some parameters, maintaining population diversity and preventing premature convergence to local optima. After many iterations, the entire population gradually evolves toward being able to complete the task.

It's worth noting that genetic algorithms are just one member of the evolutionary computation family. Others include Evolution Strategies (ES) — OpenAI's 2017 paper demonstrated that simple ES methods could rival deep reinforcement learning on Atari games and MuJoCo control tasks, with native support for massive parallelism; and Differential Evolution (DE), which excels at continuous optimization problems. The shared advantage of these methods is that they don't require gradient information. For control problems like robotic arms that involve complex physical interactions and sparse rewards, traditional backpropagation is often difficult to apply directly — because the physics simulator may not be differentiable, or because the reward landscape contains large flat regions that cause vanishing gradients — and genetic algorithms, as a black-box optimization approach, can neatly sidestep these obstacles.

MLP Neural Networks: The Brain Behind Robotic Arm Decisions

In this architecture, the MLP neural network serves as the robotic arm's "brain." MLP (Multi-Layer Perceptron) is the most classic feedforward neural network structure, consisting of an input layer, one or more hidden layers, and an output layer. Layers are fully connected, with hidden layers typically using nonlinear activation functions like ReLU or Tanh. Although surpassed by CNN and Transformer architectures in image and sequence processing, MLP remains the go-to choice for low-dimensional control tasks — it's simple in structure, fast at inference, and has a manageable parameter count, making it ideal for embedding as a real-time controller within a physics simulation loop.

In this project, the MLP receives the current state (such as joint angles and target position) as input and outputs control signals (such as joint movement directions or torques). The network's weight parameters are precisely what the genetic algorithm is optimizing.

In other words, the genetic algorithm is responsible for "evolving" an excellent set of network weights so the MLP can correctly map sensor inputs to effective actions. This "evolution + neural network" combination is academically known as Neuroevolution, an important alternative to reinforcement learning. The history of neuroevolution dates back to the 1990s, with one of the most influential works being Kenneth Stanley's NEAT (NeuroEvolution of Augmenting Topologies) algorithm, which evolves not only network weights but also network topology. In recent years, research from Uber AI Labs has shown that large-scale neuroevolution can rival mainstream deep reinforcement learning algorithms like PPO and SAC in certain scenarios, especially in "deceptive" environments with extremely sparse reward signals that require extensive exploration. Neuroevolution often performs better in these cases because it naturally maintains population diversity and is less likely to get trapped in local optima.

Input Design: The Key Factor Determining Model Convergence

One highly instructive detail from the author's experience: his initial attempts all ended in failure. It wasn't until Codex optimized the inputs that the evolutionary process rapidly converged and the robotic arm quickly learned to reach its target.

This confirms an often-underestimated truth in machine learning — feature engineering and state representation are frequently more important than the algorithm itself. The exact same genetic algorithm and network structure can achieve a qualitative leap simply by changing the form of input fed to the network.

In the specific context of robotic arm control, input representation choices involve many considerations. For example, the choice of coordinate system is crucial: if the target's absolute position in world coordinates is used as input, the network must implicitly learn the relationship between "where the end effector currently is" and "where the target is"; but if you switch to the relative coordinates (difference vector) from the end effector to the target, the learning task becomes much simpler — the network only needs to learn "how to eliminate this offset." Similarly, joint angles can be represented as raw radians or as sin/cos pairs to eliminate periodic discontinuity (for example, 359° and 1° are numerically far apart but physically nearly identical). Additionally, normalization ensures that input dimensions are within similar numerical ranges, preventing some dimensions from dominating the learning process due to excessively large values. Sometimes adding velocity information (joint angular velocities, end effector movement speed) can also significantly improve control performance, as it provides the network with dynamic information that enables it to predict future states.

Even in the era of deep learning's "end-to-end learning," these principles of input design remain critically important. While deep networks can theoretically extract features automatically from raw data, when samples are limited or the search space is vast (as with the limited population size in this project's genetic algorithm), carefully designed input representations can drastically shrink the search space and accelerate convergence.

For any developer working with reinforcement learning or evolutionary computation, this is a lesson worth remembering: when a model stubbornly refuses to converge, rather than blindly tweaking hyperparameters, first examine whether the input state representation is reasonable.

A New Paradigm in AI-Assisted Programming: From Solo Development to Human-AI Collaboration

Notably, the author explicitly stated that the entire project was "vibecoded" — completed through conversational collaboration with ChatGPT and Codex. He even specifically mentioned that a particular version of Codex helped him accomplish the critical input optimization.

The concept of "vibecoding" was first coined by Andrej Karpathy (former Tesla AI Director and OpenAI founding member) in early 2025. It refers to a development approach where developers no longer write code line by line, but instead describe their intent in natural language, let AI generate the code, and only need to "vibe" with whether the code is heading in the right direction. This paradigm disrupts the traditional software development workflow — the developer's core competency shifts from "writing code" to "describing problems, evaluating solutions, and guiding iteration."

The current ecosystem of AI programming tools is already quite mature: GitHub Copilot provides line-level and function-level code completion, IDEs like Cursor and Windsurf deeply integrate AI into the development environment, while OpenAI's Codex (a cloud-based programming agent built on ChatGPT) and Claude Code can execute more complex multi-step programming tasks, including understanding codebases, debugging errors, and even proactively suggesting architectural improvements.

This reflects a deeper trend in AI-assisted programming: developers are no longer fighting alone but treating AI as a collaborative partner to jointly explore the problem space. AI doesn't just help write code — it can provide insights at critical debugging and optimization junctures, just as Codex identified the input representation problem and proposed improvements in this project. For experimental projects involving extensive trial and error, AI assistants can significantly lower barriers and accelerate iteration, enabling individual developers to reach frontiers that previously required entire teams. This "democratization" effect is profoundly reshaping the landscape of technological innovation, as more and more cutting-edge experiments emerge not from large laboratories but from independent developer communities around the world.

The Next Challenge: From Robotic Arms to 3D Walking Robots

The author revealed that his next goal is to evolve a 3D walking robot (Walker), though he hasn't succeeded yet.

This is not surprising. Going from robotic arm reaching to bipedal/multi-legged walking represents an order-of-magnitude leap in difficulty. The robotic arm reaching task is essentially an Inverse Kinematics problem — given a target position, calculate the joint angles to bring the end effector to the target. The base is fixed, and the system's stability is inherently guaranteed. Walking is an entirely different matter, involving dynamic balance — the robot must continuously maintain its center of gravity within the support polygon during movement (or use Zero Moment Point (ZMP) theory to maintain stability during dynamic walking), requiring the controller to make precisely coordinated decisions at every time step.

The core challenges of walking tasks include: continuous gait coordination — all joints must move in sync according to precise timing patterns, where any timing deviation in a single joint can cause a fall; ground contact dynamics — collisions between feet and ground, and the direction and magnitude of friction forces change abruptly, creating a non-smooth, discontinuous dynamical system; reward signal sparsity and delay — the robot may need dozens of correct actions in continuous combination just to take one stable step, and all intermediate states before that receive virtually no positive feedback.

DeepMind's classic 2017 paper Emergence of Locomotion Behaviours in Rich Environments showcased stunning results of training various natural walking gaits using distributed PPO, but behind this was an enormous computational investment of thousands of parallel environments and billions of interaction steps. The classic "BipedalWalker" environment in OpenAI Gym is notoriously difficult. To tackle such problems with neuroevolution typically requires more sophisticated fitness function design (for example, rewarding not just forward distance but also maintaining uprightness, energy efficiency, gait symmetry, and other intermediate metrics), Curriculum Learning strategies (starting with flat ground walking and gradually introducing slopes, obstacles, and more complex terrain), as well as larger population sizes and computational budgets. Additionally, behavioral diversity preservation (such as the Novelty Search method) has been proven highly effective in these highly deceptive problems — it encourages the population to explore different behavioral patterns rather than all converging on a seemingly decent but actually locally optimal strategy.

Conclusion: Big Lessons from a Small Project

This personal project from the Reddit community may be small in scale, but it vividly demonstrates the power of combining evolutionary computation with neural networks. It reminds us that:

• Neuroevolution is a powerful tool for solving sparse-reward control problems, offering a viable alternative path where traditional gradient methods fail;
• The quality of input representation often determines project success or failure — carefully designed state space representations can transform a seemingly unsolvable problem into an easily converging optimization task;
• AI-assisted programming is empowering individual developers to conduct cutting-edge exploration — the vibecoding paradigm lowers experimental barriers, enabling more people to participate in technological innovation.

Regardless of whether the 3D walker ultimately succeeds, this spirit of openly sharing, learning by doing, and exploring within the open-source community is itself one of the most valuable assets of the AI era.