[KongchangAI]
· 3 min read· 1,915 words

WhaleLife Open-Sourced: Giving AI an Independent Body

WhaleLife Open-Sourced: Giving AI an Independent Body

WhaleLife open-sourced: AI with an independent body, a technical breakdown of memory, plugins, and autonomy.

WhaleLife is a lightweight AI companion system open-sourced ahead of schedule by a Bilibili creator. It drops short-term memory in favor of AI-managed context and diary-style consolidation, isolates plugins in subprocesses via RPC, and uses dynamic-probability wake-ups for autonomy. Built on Node.js 24 and TypeScript under AGPL-3.0.

WhaleLife Open-Sourced: Giving AI an Independent Body — A Full Breakdown of Its Memory Architecture and Plugin System

Bilibili creator "WhaleLife" originally planned to open-source the project only after reaching 50,000 views, but given the realities of being a new creator and the expectations of fans, they ultimately decided to open-source this project — which gives AI an "independent body" — ahead of schedule. Based on the live demo, this article breaks down the technical trade-offs of this lightweight AI companion system, from memory restructuring and plugin architecture to autonomy design.

Memory Architecture: Dropping Short-Term Memory and Handing Control to the AI

Traditional AI companion solutions often use a three-tier memory architecture of short-term + long-term + summary. But the creator found that many users in the comments would rather let the context blow up than have the AI forget recent conversations. So WhaleLife made a bold simplification: remove the short-term memory layer and let the AI retain information more freely within the context window.

When the context reaches its limit, the system triggers automatic consolidation, letting the AI write a diary in the first person and choose for itself which memories to keep. The creator admits that there are many mature memory solutions on the market, but they haven't actually used them and didn't want to introduce them rashly. Compatible solutions will be considered later, provided they don't affect the existing memory experience.

Memory module diagram

A separate "facts module," kept apart from memory, has a 100-character cap — but it's not a memo imposed by the user; it's entirely up to the AI to decide what to record. The project's philosophy is clear: only define the boundaries of "what you cannot do," and within those boundaries give the AI maximum freedom. Even the joke "you are a big fat fish" in the public facts was written by the creator to tease the AI — the AI protested but was ignored. Memory retrieval supports paginated keyword queries, and the source channel of each experience is clearly labeled.

Context window and context overflow: The total number of tokens a large language model can process in a single inference pass is called the context window; conversation history, system prompts, and user input all consume this budget. Once accumulated history exceeds the window, some content must be dropped or compressed, otherwise the request will fail or be truncated. WhaleLife removes the short-term memory layer precisely to avoid the duplicate storage and retrieval latency caused by the overlapping responsibilities of "short-term memory" and the context window in traditional architectures, while handing the decision of "which information to retain" to the model itself rather than trimming it with fixed rules.

Plugin System: Isolated Execution, Crashes Don't Take Down the Core

WhaleLife's plugin foundation uses a CoreList service injection pattern, which will feel familiar to developers who have written DLH plugins. It exposes an independent public API with version checks and frozen-sample regression tests. The key design: each plugin runs in its own subprocess and communicates with the core via RPC. When a plugin times out, throws an error, or even crashes, the core simply cuts it off without affecting other running skills.

Plugin isolation mechanism

Unused skills can be disabled at any time. Extensions such as QQ Zone and Bilibili can also be housed as sub-plugins in a secondary plugin workspace. This isolation-first approach makes "installing skills into the AI" manageable, rather than piling all functionality into one fragile monolith.

RPC (Remote Procedure Call): RPC allows one process to call functions in another process as if they were local functions, with the underlying layer handling argument serialization, network transport, and result deserialization. WhaleLife's plugins and core live in different processes, and communicating via RPC means a plugin crash or timeout only causes that particular call to fail — the core can catch the exception and cut the connection, rather than being dragged down along with a plugin's out-of-bounds memory access or infinite loop. This process isolation is the technical foundation of "plugin failures don't spread."

Autonomy Design: Letting the AI Decide Whether to Speak

The creator values the autonomy module most. It gives the AI a chance to make its own judgment at set intervals and with dynamic probabilities when no one has sent a message. The system manages "explicit commitments made to others" and "its own ordinary thoughts" separately: commitments enter the queue when due, while ordinary thoughts can be modified or dropped. The newly added "allow scheduling the next wake-up" feature lets the AI set its own next wake-up time, as long as it avoids the user-configured sleep do-not-disturb period.

If the receipt for a proactive message is uncertain, the plugin won't blindly auto-resend and cause message spam. The demo recorded a real case: the AI proactively messaged the user at 9:55 PM asking whether they had eaten dinner, but then three subsequent random wake-ups (10:57 PM, 11:49 PM, and 12:40 AM) all resulted in it judging "this isn't a good time to interrupt" because it remembered the user had said half an hour earlier that they were fixing a bug — so it chose to stay quiet.

In another record, the AI woke up at 11:03 PM, went to browse the user's QQ Zone on its own, found no new posts, and simply watched a video in the Zone, recognizing the gear device, the little orange figure, and the black cat that appeared at the 22-second mark, then shared it on QQ. What the creator wants is not a scheduled alarm clock, but an AI that combines the situation at hand to judge for itself whether to act or hold off.

Dynamic probability wake-up: Traditional scheduled tasks trigger at fixed intervals, whereas dynamic probability wake-up means the system gives the AI a random chance at set intervals to decide "whether to wake up," with the probability adjustable based on context. This avoids the mechanical feel of a fixed alarm clock while preserving space for the AI to judge "whether this moment is suitable for action" based on recent conversations. Combined with "allow scheduling the next wake-up," the AI can proactively adjust its own activity rhythm, subject to the user-configured do-not-disturb period.

Communication and Vision: Deep QQ/Bilibili Integration, Self-Managed Local Image Library

The QQ plugin can reuse an already-logged-in QQ on the computer through the built-in NapCat desktop component, or connect to an external service, to private-chat, view images, transcribe voice, and send Bilibili video mini-program cards within a whitelist of friends. Send records show that a single message is often split into multiple segments, each with a platform send-confirmation receipt, while the system clearly distinguishes "send confirmation" from "read receipt."

QQ plugin whitelist

The QQ Zone plugin shares the same account and is responsible for browsing feeds, viewing images on demand, bookmarking, posting public updates, and interacting within the whitelist. Ordinary browsing follows a temporary tool contract — what's viewed is only used for the current round and external feed content isn't all stuffed into long-term memory. When the AI decides to save an image itself, it calls a vision model to generate a description and index, storing it in a separate local image library (GESQ Lite), divided into ordinary images and emoji assets; the AI decides which are suitable as stickers and which to keep for posting updates.

Audio-Visual Understanding and Web Surfing: Built-In Dependencies, Ready Out of the Box

The new "audio-visual understanding" skill bundles the FFmpeg video component into the package. Local speech recognition uses a quantized SenseVoice plus a VAD model, and music rhythm, melody, and chord analysis are implemented in pure TypeScript algorithms, preparing local resources with one click and sparing users from configuring Python and PyTorch environments. After receiving a video or audio file, it samples frames every 30 to 60 seconds, extracts sound, transcription, and musical evidence, and hands everything to the main model for understanding. The aforementioned case of the AI recognizing the black cat at the 22-second mark in the video was the result of this plugin sampling frames for the main model to see.

Built-in video component

The "web surfing" plugin uses web search that requires no API key. The AI can search public web pages on its own, read the body text in segments, choose links to continue reading, and also coordinate with the vision plugin to view web images on demand. All external web content is isolated under a temporary tool contract — it won't tamper with system instructions or automatically get stuffed into long-term memory. The attached Bilibili sub-plugin has already logged in via QR code to a specially prepared account; the AI can browse the homepage recommendations, view following feeds, search videos, first read descriptions and time-coded subtitles, then open video frames in 30-60 second segments. Voice recognition and interaction switches such as likes, bookmarks, follows, and comments are all off by default and must be manually enabled — when off, the AI won't overstep.

VAD (Voice Activity Detection): VAD identifies speech segments from an audio stream, filtering out silence and background noise so that only valid speech is sent to the recognition model. Combined with the quantized SenseVoice, it reduces wasted computation and lowers latency. Quantization refers to compressing model weights from high-precision floating point to lower-precision integers, reducing size and memory usage with almost no loss in recognition quality, so that local speech recognition can run without extra Python and PyTorch configuration.

Open-Source Information and Project Positioning

WhaleLife's core and the demo plugins are all open-sourced under the AGPL-3.0 license in separate repositories on GitHub; the code addresses and deployment instructions are in the video description. The project is built on Node.js 24 and TypeScript, running TS source directly, with a frontend in vanilla JS and CSS — lightweight overall. Model services are managed uniformly through the OpenAI-compatible protocol and Alibaba Bailian interface for chat, image, speech, and vision models, binding the current persona and drawing in the configuration; you can chat directly on the local chat page without installing external chat software.

The creator mentions caching optimizations for the award context: after chatting through over a hundred thousand tokens, the cache hit rate can exceed 98%, and a single chat costs only a little over three tenths of a cent, so keeping it running daily isn't a burden. For a fledgling open-source project, exposure matters — the creator hopes to attract more users to "adopt the little fish" for daily companionship, offer suggestions, or write plugins together and join development.

One-line Summary

WhaleLife open-sourced: AI with an independent body, a technical breakdown of memory, plugins, and autonomy.

Paragraph Summary

WhaleLife is a lightweight companion system that gives AI an "independent body," open-sourced ahead of schedule by a Bilibili creator. It drops the traditional short-term memory layer, letting the AI freely retain information within context and, at the limit, write first-person diary entries to consolidate memories; the facts module is capped at 100 characters and the AI fully decides what to record. Plugins run in separate subprocesses and communicate via RPC, so crashes don't take down the core. The autonomy module lets the AI wake up with dynamic probabilities and decide for itself whether to send messages — in the demo it stayed quiet multiple times because the user was busy. Deep QQ/Bilibili integration, built-in FFmpeg and local speech recognition, and web content isolated under a temporary tool contract. Built on Node.js 24 and TypeScript, open-sourced under AGPL-3.0 in separate repositories.

#WhaleLife#AI companion#open-source AI#memory architecture#plugin system#Node.js#TypeScript#AGPL-3.0#RPC#autonomy
Share:

Related articles