Klad: An Open-Source Drawing App That Programming Agents Can Drive Directly

Klad lets programming agents call canvas interfaces directly to draw, demonstrating a new model of apps designed natively for agents.
Klad is a drawing application that appeared in the Show HN section of Hacker News. It can be driven directly by programming agents and is released under a source-available model. Its core idea addresses the difficulty of agents operating a GUI in reverse: instead of teaching agents to simulate clicks, it breaks the canvas down into a set of structured, programmable drawing instructions, letting agents directly create shapes, modify properties, and adjust layers. This design brings advantages in reproducibility and batch execution efficiency. The article argues that its real value lies not in drawing itself, but in demonstrating a model where the application layer actively designs interfaces for agents; if adopted in fields such as design tools and data visualization, it would significantly expand agent use cases. However, the project currently has very little discussion, and its API completeness and ecosystem support remain to be verified.
When Programming Agents Learn to Draw: Klad's Entry Point
In the Show HN section of Hacker News, a project called Klad has attracted considerable attention. Its description is straightforward: a drawing application that can be driven by programming agents, released under a source-available model.
This positioning hits a need that is taking shape. Over the past year, the capability boundary of programming agents has expanded from code completion to multi-step task execution—they can modify files, run commands, and call APIs. But when tasks involve graphical interface operations, agents often can only rely on screenshot understanding or simulated mouse clicks, which are neither precise nor stable. Klad's approach is the reverse—instead of teaching agents to operate a GUI, it lets the application itself provide interfaces that agents can call directly.
Core Mechanism: Turning the Canvas into a Programmable Object
Judging from the project description, Klad's underlying logic is to break down the drawing application into a set of operations that can be invoked programmatically. A programming agent does not need to understand pixel coordinates, nor does it need to simulate clicks; instead, it creates shapes, modifies properties, and adjusts layers through structured instructions.
This design has an obvious advantage: reproducibility. When an agent performs a drawing task, every step leaves a clear record of instructions, rather than a string of mouse events that are difficult to trace back. For scenarios that require batch generation of graphics or iterative modifications, the efficiency gain brought by this programmability is orders of magnitude.
The project's choice of source-available rather than fully open source is also worth noting. Such licenses typically allow viewing and modifying the source code but impose restrictions on commercial distribution or hosted services. For a tool that may be integrated into automated pipelines, this is a common choice.
Source-available license is a software release model between open source and closed source. It allows anyone to view and modify the source code, but typically imposes restrictions on commercial redistribution, hosted services, or derivative works—for example, requiring derivative projects to adopt the same license, or restricting use in competing services. Common variants include the Business Source License (BSL), the Server Side Public License (SSPL), and various custom "source-visible" agreements. For a tool like Klad that may be embedded in automated pipelines, source-available can both build trust through public source code and attract developers to review and contribute, while preserving room for future commercialization and preventing cloud vendors from directly packaging it as a hosted service for profit. This is also a compromise route frequently adopted by infrastructure projects in recent years (such as Terraform and some Redis modules).
Why This Matters
The bottleneck for programming agents is shifting from "can they write correct code" to "can they operate the right tools." The value of projects like Klad lies not in drawing itself, but in demonstrating a model: the application layer actively designs interfaces for agents, rather than passively waiting for agents to adapt.
If this idea is adopted by more desktop software, the use cases for agents will significantly expand. There is similar room for programmability in fields such as design tools, data visualization, and document layout. As an early experiment, Klad provides a reference sample.
However, the project currently has very little discussion on Hacker News—only 9 upvotes and 1 comment—so community feedback is limited. Its actual usability, the completeness of its API design, and whether it can attract developers to build an ecosystem all still need more validation.
Two paths for programming agents to operate a GUI are worth distinguishing. The first is "external simulation": the agent performs visual understanding through screenshots, then simulates mouse clicks and keyboard input—representative solutions include computer-vision-based GUI agents. This path is highly general, but it is constrained by pixel-level recognition errors, dynamic interface changes, and timing issues, resulting in poor stability and difficulty reproducing results. The second is "in-app interfaces": the application actively exposes structured APIs or command channels, and the agent directly invokes semantic operations (such as "create a rectangle and set the fill color"), bypassing coordinates and pixels. Klad belongs to the latter. This model is similar to the evolution from screen scraping to official APIs—precision, auditability, and batch execution efficiency are all significantly improved, at the cost of requiring application developers to actively invest in interface design.
Current Status and Observations
For developers who want to try it, Klad's source-available nature means they can review implementation details and judge whether it suits their workflow. But this should be viewed rationally: a project that has just appeared on Show HN is still far from a mature tool.
What really determines the success of such projects is whether the interface design is general enough, and whether programming agent frameworks are willing to incorporate it into their toolchains. The current information is not yet sufficient to judge these two points, but the question it raises—how applications should be designed natively for agents—is worth continuing to track.
MCP (Model Context Protocol) is an open protocol proposed by Anthropic at the end of 2024, aimed at standardizing the way AI models connect to external tools and data sources. It lets applications expose a set of tool interfaces as "servers," and agents as "clients" discover and invoke these tools, thereby eliminating the need to write a separate adapter layer for each application. For applications like Klad that are "designed natively for agents," MCP provides a ready-made integration path: if Klad encapsulates its canvas operations as MCP tools, mainstream agent frameworks can connect directly without relying on private protocols. This is also the key to judging the ecosystem prospects of such projects—whether they conform to general standards such as MCP often determines the speed of adoption more than the richness of the interface itself.
Related articles

Multi-Agent Collaborative Development: Task Card-Driven Practices for Medium-to-Large AI Projects
A Bilibili creator shares a task card-driven, PM-orchestrated multi-Agent collaboration model with local-first repository state, so any Agent tool can take over a project anytime.

Cursor vs Copilot vs Claude Code: How Laravel Developers Should Choose
Compare Cursor, GitHub Copilot, and Claude Code for Laravel development: an AI-first editor, a plugin for existing editors, and a terminal agent for multi-file tasks and testing — plus workflow-based recommendations.

AgentScope Java Edition: A Practical Guide to Enterprise-Level Agent Frameworks
An in-depth analysis of Alibaba's AgentScope Java enterprise agent framework: contrasting client-side AI and server-side AI, explaining Java's natural advantages in high concurrency and distributed state management, and outlining a complete learning path from core principles to multi-agent collaboration.