Open-Source AI Desktop Robot: Built with a Laptop + Android + Pico

An open-source desktop robot project called PAL9000 uses a laptop, an Android phone, and a Raspberry Pi Pico to assemble a "desktop companion" that can see, hear, speak, and turn its head to follow people. The author shared the complete architecture on Reddit and raised several open questions about making the robot feel more "alive."

Three Pieces of Hardware, Each with Its Own Role

According to the original Reddit post, the core idea of this robot is to reuse the computing devices you already have on hand:

• Raspberry Pi Pico W: Directly drives the two servos on the 2-DOF pan-tilt head, receives angle values sent over serial, and runs EMA smoothing to avoid jerky motion.
• MacBook: Handles the two heavy-load pipelines of vision and conversation. It uses MediaPipe plus AprilTag markers attached to people's bodies to determine people's positions from the camera feed, and switches attention among people wearing specific tags; then it converts the 2D coordinates into pan-tilt angles through two LOBF (linear fits) generated by a calibration script, and sends them to the Pico over serial.
• Android phone: Serves as the robot's "face," running an animated face on screen whose pupils follow the movement of human faces.

The conversation part also runs on the laptop: silero-VAD for voice activity detection, whisper-cpp for transcription, and llama-cpp for local inference with the gemma4:12b Q4 quantized model. The entire pipeline does not depend on cloud-based large models, so privacy and latency are under your own control.

AprilTag is a visual fiducial marker system developed by the APRIL lab at the University of Michigan. It looks similar to a QR code but is designed specifically for fast, robust pose estimation. Each tag has a unique ID, and after the camera recognizes a tag, it can calculate the tag's 3D position and orientation relative to the camera. In this project, the author attaches different AprilTags to different people. After MediaPipe on the MacBook detects a face, it identifies the nearby AprilTag ID to distinguish "who this is," thereby enabling attention switching among different wearers. Compared with face recognition, the AprilTag approach requires no model training, is unaffected by lighting and angle, and is extremely low in computation, making it very suitable for personal robotics projects.

LOBF (linear fit) here refers to two linear mapping functions obtained through a calibration script: one maps the horizontal pixel coordinates in the frame to the pan servo angle of the pan-tilt head, and the other maps the vertical pixel coordinates to the tilt angle. Because there are installation offsets and lens distortion between the camera and the pan-tilt head, direct conversion using geometric formulas would introduce errors. The author collects several sets of "pixel coordinate - actual angle" samples and fits straight-line equations, so that at runtime the center of a 2D detection box can be quickly converted into servo commands. This method is simple and effective, but its accuracy is limited by calibration quality and field of view.

Modularity Is the Biggest Highlight

The author jokes that "modular" is a euphemism for "everywhere," but this is precisely what lowers the barrier for others to reproduce it. You can run only the face and eye tracking in the browser, run only the conversation server (audio output requires minor changes), or run only the robot control program to make it turn its head and stare at you. For people who want to get started with robotics without having to digest the entire system at once, this separable architecture has more learning value than a monolithic finished product.

silero-VAD is a lightweight voice activity detection (VAD) model used to determine which segments of an audio stream contain human speech. It is based on a neural network, but the model is extremely small (about 1 MB) and can run in real time on a CPU. It is often used at the front end of voice assistants to reduce unnecessary speech transcription and inference overhead. In this project, silero-VAD continuously listens to microphone input. Once it detects that the user has started speaking, it triggers whisper-cpp for transcription; when the user finishes speaking, it automatically cuts off, avoiding sending silence or noise into the subsequent pipeline. This "detect first, transcribe later" pipeline can significantly reduce latency and compute usage.

whisper-cpp is a C++ port of OpenAI's Whisper speech recognition model, supporting local execution on CPU without a GPU or cloud API. It retains Whisper's multilingual recognition capability while compressing model size through quantization techniques, making real-time transcription possible on an ordinary laptop. The author chose whisper-cpp rather than a cloud speech service to ensure privacy and offline availability, consistent with the project's overall design philosophy that "data does not leave the local device."

Anthropomorphism: The Most Interesting Part of the Project

The author explicitly states that he is particularly interested in "making the robot anthropomorphic." He openly solicits two kinds of ideas: on-screen animations and physical motions at the servo level, to make the robot seem more alive. He is also struggling with a specific question - is 2 degrees of freedom enough to support this sense of life, or should another servo be added?

The answer to this question depends on what you want to express. A 2DOF pan-tilt head can produce basic social cues such as gazing, nodding, and head tilting, but it lacks motions like "shrugging" and "hunching the neck" that require additional axes. If the goal is only "it is looking at me," 2DOF is enough; if you want to convey emotional states, a third axis would significantly expand the expressive space. The original post does not provide a conclusion, and this in itself is a design trade-off worth community discussion.

A 2-DOF (2 degrees of freedom) pan-tilt head refers to a mechanical structure that can rotate around only two axes, usually horizontal (yaw) and vertical (pitch). It can point the camera or screen in any direction, but cannot perform roll (tilt) motion. In social robots, 2DOF is sufficient for basic nonverbal cues such as "gazing," "nodding," and "head tilting," because these motions mainly rely on combinations of yaw and pitch. However, motions involving changes in body posture, such as "shrugging" and "hunching the neck," require additional degrees of freedom (such as a roll axis or a translation axis) to express. The author's dilemma over whether to add a third servo is essentially a trade-off between expressiveness and mechanical complexity: a third axis can significantly expand the space of emotional expression, but it also increases wiring, power supply, and control difficulty.

Real Dilemmas in Engineering Details

The author admits that the wiring is "a bit messy," hopes to make an enclosure to organize things neatly, and asks the community for storage tips. He has also considered making a custom PCB but feels that, given his current robotics experience, it is too early, and asks, "Or should I just jump into the deep end?"

Such problems are very typical in personal robotics projects: after the functions work, mechanical structure and electrical integration are often the parts that really consume time. For newcomers, making the enclosure first and the PCB later is the safer path, because only after the enclosure is finalized are there grounds for determining board dimensions and interface positions.

From a School Assignment to an Open-Source Project

This project started as a school assignment, and the author admits he "went a bit overboard." The entire project is open source and emphasizes "non-vibecoded" (not generated casually by AI). The code is hosted on GitHub (SpaceMonkeyAlfa/PAL9000-Desktop-Robot), and the development process is documented on the personal blog spacemonkeyalfa.com. The author welcomes forks and use for any purpose.

For people who want to get hands-on, the most valuable thing about this project is not any single technology, but that it demonstrates a low-cost path: use an existing laptop to run vision and the LLM, use a phone as the display, and use a microcontroller for real-time motion. Having three devices each do what they are good at is more realistic than forcing everything into a single computing box.

If you happen to have an idle Android phone and a Pico on hand, this project is worth taking apart and looking at.

One-line Summary

Open-source desktop robot PAL9000 uses a laptop, an Android phone, and a Pico W to assemble a desktop companion that can see, hear, and follow people.

Paragraph Summary

PAL9000 is an open-source desktop robot project. The author uses a MacBook for the vision and conversation pipelines (MediaPipe + AprilTag localization, silero-VAD/whisper-cpp/llama-cpp local inference), a Raspberry Pi Pico W to drive a 2-DOF servo pan-tilt head, and an Android phone as an animated face. The project is highly modular, and each module can run independently, lowering the barrier to reproduction. The author openly solicits anthropomorphic animation and physical motion ideas, and discusses whether 2DOF is enough to express a sense of life and whether a third servo should be added. In engineering terms, wiring organization, enclosure design, and whether it is too early to make a custom PCB are typical dilemmas. The project is open source and emphasizes non-AI-generated code, demonstrating a low-cost path that reuses existing devices.