LUCI Desktop: Giving AI Agents a Memory of Your Screen

A webpage that flashed by, a decision made in a meeting — both are often completely forgotten a few hours later. That's exactly the problem LUCI Desktop aims to solve: it turns everything that has happened on your screen into long-term memory that AI Agents can draw on.

The product topped Product Hunt's daily rankings with 195 upvotes and 25 comments, categorized under productivity and artificial intelligence. Developers RunzeYang and Shawn Shen position it as a local context layer that spans applications, built to serve increasingly popular AI coding and assistant tools like Claude Code, Cursor, and Codex.

What It Actually Does

LUCI's core functionality comes down to two things: recording your screen history and transcribing your meetings. All of this data is stored locally on your device. When you need to track down a page you forgot to bookmark, or can't remember a decision made on a call, connected AI Agents can search through this local memory to retrieve it for you.

The idea builds on the "personal memory store" concept that has been discussed repeatedly over the past few years, but LUCI places its emphasis on Agent collaboration. Rather than having you manually dig through chat logs, it enables tools like Claude and Cursor to proactively reference context you've previously seen or heard while executing tasks. In other words, it tries to patch the biggest gap in today's AI assistants: the lack of persistent memory about a user's personal working context.

Cross-App Capture Without Connectors

One of the more notable aspects of LUCI is its approach to context capture. Most mainstream AI integrations rely heavily on connectors: linking Notion requires a plugin, connecting a calendar requires API authorization, and wiring up email involves yet another workflow — forcing users to configure each application separately.

LUCI takes the opposite approach — capturing directly at the screen level. Whether content appears in a browser, a chat app, or a document editor, it captures everything without needing per-app connectors. This "watching the screen" style of collection dramatically lowers the barrier to entry and also covers scenarios where the underlying application has no open API. The tradeoff is continuous access to screen content, which is precisely why local storage becomes a critical architectural choice.

Screen-level capture works by continuously taking screen frames and running OCR recognition and semantic indexing on the image content — essentially producing a searchable "documentary" of everything the user sees. Its key advantage is universality: if a human eye can see it, it can be recorded, naturally bypassing the data silos between different applications. macOS's Recall competitor sparked significant controversy primarily over the security of screenshot data. LUCI's choice to keep data local is a direct response to those concerns. It's also worth noting that screen-level capture generates more redundant data than API integration (ads, system notifications, etc.), so how well the indexing stage filters and denoises this content directly determines the quality of subsequent retrieval.

Local-First and On-Device AI

Keeping all screen history and meeting recordings on the local device is something LUCI emphasizes repeatedly. This isn't just a privacy selling point — it also shapes the technical architecture.

The product integrates Microsoft Foundry Local to handle transcription and daily summaries on-device. Speech-to-text runs directly on the local machine without uploading audio to the cloud, and daily activity is automatically compiled into summaries. For users dealing with sensitive information like meeting content and internal decisions, keeping data on-device is a genuine requirement — and the maturation of on-device models has made this kind of design feasible.

LUCI is currently free on both Mac and Windows, which lowers the barrier to trying it out.

Microsoft Foundry Local is Microsoft's on-device AI inference framework, allowing developers to deploy language models directly on user devices to perform inference without network requests. It supports multiple mainstream model formats and is performance-optimized for NPUs (Neural Processing Units) and GPUs on the Windows platform. The rise of on-device inference (on-device inference) is driven by advances in model compression over the past two years — techniques like quantization and distillation have made it possible to run billion-parameter models on consumer hardware. For applications like LUCI that need to continuously process audio and text in the background, on-device models mean lower latency and more controllable resource consumption, though they do place certain demands on user hardware, particularly in terms of memory and storage.

A Few Observations

LUCI represents a clear direction in the evolution of AI tools: moving from "answering what you ask right now" toward "understanding what you've been working on over time." When an Agent can draw on your screen memory, the assistance it provides becomes much more aligned with your actual workflow rather than starting from scratch every time.

That said, screen-recording-based memory solutions inherently carry privacy and trust concerns. Even with local storage, whether users are willing to let a tool continuously "watch" their screen remains a critical factor in whether this category of product can scale. LUCI answers that question with local storage and on-device AI — but the real-world effectiveness still needs longer-term usage to verify. Current product information stays largely at the level of capability descriptions; actual performance on retrieval accuracy, resource usage, and similar metrics has yet to be tested.