ESP32-S3 Desktop Robot Pamk: Custom Wake Word + Gemini as a Sticky Note Replacement

When Sticky Notes Get Replaced by a Talking Desktop Robot

Scattered sticky notes during exam prep are a familiar struggle — reminders you forget to check, notes you can't find when you need them. A French student solved this problem with a palm-sized desktop robot called Pamk, built on the ESP32-S3. It understands voice commands, manages tasks and schedules, and proactively reminds you at the right moment. The entire project — from the enclosure to the circuitry — was almost entirely built by the creator himself.

Unlike smart assistants that rely on a phone app, Pamk's interaction model is refreshingly direct: just say "Hey Pamk" to wake it up and start a conversation. No app to open, no screen to unlock. That "say it and walk away" experience is exactly the kind of lightweight feel a desktop productivity tool needs.

Core Features: A Voice-Driven Task Manager

Pamk is designed as a desktop companion focused on personal organization. Its main capabilities include:

• Pure voice interaction: Activated by the custom wake word "Hey Pamk" — no app required at any point;
• Task and schedule management: Dictate to-do items and events, with reminders triggered at set times;
• Pomodoro timer: A built-in Pomodoro mode displays a progress ring around the robot's "face" on screen, giving an intuitive visual of remaining time;
• Swappable face panels: 3D-printed face panels attach magnetically and can be swapped out at any time.

The progress ring detail is worth highlighting — it transforms an abstract countdown into a visual animation wrapped around the robot's face, making focused time feel tangible. It's a more emotionally resonant experience than a simple numeric timer, and perfectly suited to a desktop companion.

The Pomodoro Technique was developed by Francesco Cirillo in the late 1980s. The core rule is simple: break work into 25-minute focused blocks (one "pomodoro"), followed by a 5-minute break, with a longer rest after every four pomodoros. The psychological rationale is that fixed time-boxes lower the friction of starting a task, while mandatory breaks help sustain cognitive resources. Integrating a Pomodoro timer into a physical desktop device rather than a phone app helps reduce the distraction of picking up your phone to check progress — and that's the real interaction-level value of Pamk's circular progress bar: the question "how much time is left?" answers itself without any deliberate action.

Hardware and Software: A Complete Embedded Systems Build

Pamk's tech stack reflects the hallmarks of a typical ESP32 maker project, but with a notably high level of polish.

Microcontroller and Display

The heart of the system is Espressif's ESP32-S3 — an MCU with AI acceleration instructions and ample memory that has become a popular choice for voice-focused DIY projects in recent years. The display is a 280×240 resolution screen used to render the robot's expressions.

Compared to the earlier ESP32 and ESP32-S2, the ESP32-S3's most significant upgrades are its built-in vector instruction extensions (for accelerating neural network inference), larger on-chip SRAM (up to 512KB), and native USB OTG support. For voice projects, the extra memory means lightweight wake word detection models can run locally without streaming all audio to the cloud. Espressif also built the ESP-SR speech recognition framework around the ESP32-S3, which includes the ESP-WakeNet wake word engine and the ESP-MultiNet command recognition engine — a common software starting point for DIY voice assistant projects. The fact that Pamk's creator chose to train his own wake word model rather than use this off-the-shelf framework means he paid a higher engineering cost in exchange for greater customization and flexibility.

Audio Pipeline

Audio input is handled by an INMP441 I2S digital microphone, while output is driven through a MAX98357 I2S amplifier connected to a speaker. This I2S mic-plus-amp combination is a classic pairing for embedded voice projects, balancing audio quality with cost.

Speech Understanding and Synthesis

For natural language understanding, the creator integrated Google Gemini to parse spoken commands into structured tasks and schedules. For the robot's spoken responses, he used personally recorded voice clips stitched together — a less flexible approach than TTS, but one that gives Pamk a distinctly personalized character.

The wake word component deserves special attention: rather than using an off-the-shelf solution, the creator trained a custom wake word model using thousands of self-recorded audio samples. This is the highest-difficulty technical component of the entire project, and it explains how Pamk achieves a local, app-free wake experience.

Wake word training is fundamentally a Keyword Spotting (KWS) binary classification problem: the model must detect in real time whether the target word appears in a continuous audio stream, while maintaining extremely low tolerance for both false positives and false negatives. Training data needs to cover variations in speaker distance, noise environment, and individual pronunciation — which is why thousands of samples were necessary. Common lightweight KWS model architectures include DS-CNN (depthwise separable convolutional networks) and LSTM-based variants, typically running inference over MFCC (Mel-frequency cepstral coefficient) features. The entire model can be compressed to tens of kilobytes for local deployment on an MCU. The advantage of a self-trained model is that the wake word is fully private and inference runs completely offline — which creates an interesting layered architecture alongside Gemini's cloud-based understanding: local wake + cloud comprehension.

Structural Design

The enclosure is entirely 3D printed, with magnetically attachable swappable face panels that are both convenient and add a fun customization element. The creator also mentioned that a V2 version is in the works with a custom PCB, signaling that this project is maturing from prototype toward a more polished product form.

From Personal Project to Open-Source Plans

The creator shares 3D model work on MakerWorld (currently some Halloween decorations) and plans to open up crowdfunding and build materials for Pamk once sufficient interest is established. As he explained, MakerWorld only opens crowdfunding to creators with an established audience — so his goal is to build community first, then let more people replicate the robot themselves.

This "open-source first, productize gradually" path is a common growth trajectory for independent hardware creators today. For developers looking to get into voice-interactive embedded projects, Pamk offers a remarkably complete reference implementation: from microphone capture and wake word training, to cloud LLM comprehension and personalized voice output, it covers nearly every key module of an end-to-end voice assistant.

A Few Observations

What makes Pamk most compelling isn't the sophistication of its hardware — it's that the creator refined the humble goal of "replacing sticky notes" to the point of genuine usability. The custom wake word, self-recorded voice, and magnetic face-swapping all point in the same direction: making a DIY robot actually integrate into daily life rather than staying stuck in demo mode.

That said, relying on Gemini means voice understanding requires an internet connection. Privacy and offline availability are trade-offs this type of project will need to address going forward. But as a student's personal build, Pamk's engineering completeness and product thinking already surpass many comparable maker projects. Looking forward to seeing community variants once it goes open-source.

Project demo video (in French, with English subtitles): https://youtu.be/_wxieeF5Wrw