Devin AI in Practice: How a Cloud-Based AI Software Engineer Delivers Complete Projects Autonomously

When You Hand a Real Engineering Project to AI

Imagine this: you hand an AI a complete engineering project, walk away, and come back to find a working system — tested, evaluated, reviewed, and fully documented. That's not science fiction; it's what Cognition's Devin AI is actively trying to deliver.

Devin is positioned as an "autonomous software engineer for your team," with the official tagline promising end-to-end automation from problem statement to merged PR. It can run in the cloud or on-premises, conducting tests in its own browser, working through issues until a PR is ready to merge. Its workflow revolves around four core phases: understand and plan, code and test, review, and automated maintenance.

This article is based on a complete hands-on demo by a YouTube creator, documenting the full process of Devin building an Agentic RAG system from scratch — with the presenter writing virtually no code themselves, collaborating with Devin throughout.

Devin Cloud: The Browser as Your Workbench

Devin Cloud is the central platform for this demo. Unlike local installations, Devin Cloud runs everything in a cloud sandbox — a complete Linux environment with a shell, code editor, and browser — allowing Devin to independently research, write code, run test cases, and fix its own errors.

After logging into app.devin.ai, users see the cloud control panel. No installation required — just open a browser and issue instructions. The left panel offers options for automation, new sessions, reviews, and more, along with different compute tiers (Fusion, Ultra, Normal, Light) and model switching between options like Opus and GPT.

Notably, Devin Cloud includes robust environment configuration: users can create Ubuntu, macOS, or Windows virtual environments, configure security settings, connect GitHub repositories, and manage MCP connectors. Since it's fundamentally a web app, it works on mobile too — check running sessions or review results from anywhere.

The demo clarified an important point: there is no separate "Devin Cloud" installer. The web app is the cloud control panel. Whether a session is launched from the web, desktop client, or CLI, it always runs in its own isolated cloud sandbox.

From Empty Repository to Complete RAG System

The hands-on portion started with a blank GitHub repository called agentic rag system demo. The presenter first completed several key integrations: GitHub for version control (otherwise Devin proactively prompts for integration), Slack for communication tracking, and Linear for converting tickets into PRs.

For mode selection, Devin offers Agent, Ask, Plan, QA, Auto, and Deep modes. Deep mode provides deeper reasoning for trickier problems. This build used Agent mode with a GPT model and a Ubuntu virtual environment.

The prompt was comprehensive, covering every aspect of engineering standards: create a requirements.txt, use Chroma as the vector store, use OpenAI for embeddings and LLM, create an agents.md describing project structure and conventions, enforce testing, ban hardcoded secrets in code, run lint before PRs, build test skills, and cover ingestion, retrieval, an evaluation harness, and a Streamlit Web UI. The rules section emphasized small commits, one PR per work slice, writing tests for core logic, and running the app with a recorded test video before any UI-touching PR.

Upon receiving the task, Devin automatically created a plan, cloned the repository, set up Python dependencies, and created a virtual environment. Along the way, it proactively asked for decisions at key junctures — for example: "GitHub needs an initial default branch for PRs to work — should I push an initial commit?" It opened a PR for each completed slice, and the presenter merged them one by one.

Agentic RAG (Retrieval-Augmented Generation) is an advanced paradigm that introduces autonomous agent capabilities on top of standard RAG architecture. Standard RAG works by chunking external documents, vectorizing them into a database, and at query time retrieving relevant segments to feed as context to a large language model — reducing hallucinations and enabling private knowledge base Q&A. Agentic RAG adds autonomous decision-making on top: the agent can decide when to retrieve, which knowledge source to query, whether results are sufficient, whether multiple iterations are needed, and can even call external tools or decompose problems into parallel sub-queries. This demo used Chroma as the vector store (a lightweight open-source vector database), OpenAI for converting text to vectors (embeddings) and final answer generation (LLM), and Streamlit for an interactive web frontend — forming a complete engineering reference implementation of an Agentic RAG system.

Computer Use: AI That Tests Its Own UI

One of Devin's standout capabilities is "Computer Use." When enabled, Devin gains access to a full desktop environment — not just a browser. It can move the mouse, click UI elements, type on the keyboard, take screenshots, and interact with any application running on the desktop.

This means Devin can genuinely test the Streamlit app it built: launch the app, upload documents, trigger retrieval, and observe returned results and citations. In the demo, Devin uploaded a Markdown test document, issued a query, and correctly handled a case with no relevant evidence — returning "unable to find relevant evidence in the uploaded documents." The entire UI verification process was also recorded as a video and attached to the PR.

For secret management, the OpenAI API key was added to Devin's Secrets, and the app automatically reads it from environment variables — no secrets stored in the codebase. A practical security detail worth noting.

Automation, Slack, and Linear: Letting Devin Trigger Itself

Beyond manual session launches, Devin's automation features allow it to self-trigger. Trigger sources can be Slack messages, GitHub events, or any webhook. The platform provides multiple templates covering bug reports, CI/CD releases, security, project management, and more.

The presenter created a bug report automation: when a bug is posted in a channel, Devin investigates the codebase, identifies the root cause, and posts its findings in the thread when confidence is high. Devin was also @-mentioned directly in Slack with a request to "profile the embedding pipeline and fix the slowest part" — Devin acknowledged with an emoji reaction, then completed the performance analysis and bottleneck fix.

The Linear integration worked just as smoothly: @-mentioning Devin in a project with "what's the file size limit for each document type" got a response that the current limit is 200MB per file. When asked to raise it to 250MB, Devin pointed out that "the 200MB limit comes from Streamlit, not our code" — accurately identifying the source of the constraint, demonstrating real understanding of the tech stack.

Outpost, Review, and Security: Enterprise-Grade Capabilities

Devin Outpost is a distinctive feature that allows Devin sessions to run on a user's own infrastructure. The presenter created a VM with a GPU, installed the Devin CLI (experience similar to Claude Code), registered the machine as a worker, generated a token, and connected it. From there, the machine could be invoked directly from Slack to perform tasks — for example, adding a re-ranking stage to the retrieval pipeline using an open-source reranking model.

Devin Review addresses the pain point of "AI writing code faster than humans can review it," providing a complete code review platform with support for GitHub, GitHub Enterprise, and GitLab. Simply replace github.com in a repository URL with devinreview.com to trigger analysis. In the demo, it discovered a real bug: model downloads would block document ingestion (during the first download of the Flash ranking model), pinpointing the exact line number.

On the security front, the presenter highlighted two dimensions: whether Devin itself is safe to introduce into an enterprise, and whether Devin can perform security work. At the platform level, Cognition has achieved SOC 2 Type 2 certification and ISO 27001 compliance, with data encrypted in transit and at rest, and each session running in an independently isolated VM. On the functional side, deep security scanning identified several issues including injection-style attacks, which were fixed in parallel by spinning up three separate sub-sessions.

SOC 2 Type 2 and ISO 27001 are the two most widely adopted security compliance frameworks in enterprise software. SOC 2 (Service Organization Control 2), developed by the American Institute of CPAs, requires auditors to verify the actual operating effectiveness of a company's security controls over a period of time (typically 6–12 months) — not just reviewing written policies. This makes it a critical credential for SaaS providers demonstrating data security trustworthiness to enterprise customers. ISO 27001 is the International Organization for Standardization's standard for information security management systems, covering risk assessment, access control, incident response, and the full security management lifecycle. For enterprise teams that need to hand over codebases, API keys, and even production system access to an AI agent, these two certifications mean Cognition's data handling processes have been substantively audited by independent third parties — not merely self-attested.

Managed Devins and Knowledge Accumulation

Managed Devins is one of Devin's more advanced capabilities: an orchestrator session can decompose large tasks and distribute them to a group of managed sessions, each running in its own independently isolated VM — monitoring progress, resolving conflicts, and consolidating results. This differs from the sub-agents pattern common in tools like Claude — here these are truly independent, isolated virtual machines, not simple process forks.

To wrap up, the presenter deployed the finished Streamlit app to port 8080 via ngrok for external access, uploaded a research paper on Docling, successfully queried it, and verified end-to-end functionality. The demo also showcased MCP configuration, a plugin marketplace, custom skills, hooks, rules, and other extensibility features — as well as the Deep Wiki feature, which automatically generates comprehensive architecture documentation for a repository, including system overviews, repository layout, data flow diagrams, project configuration, and custom skill descriptions.

ngrok is a widely used tunneling tool for temporarily exposing local services to the internet: it establishes an encrypted tunnel between a local port and ngrok's cloud servers, generating a publicly accessible URL without requiring firewall configuration or a public IP address. Using ngrok to publish the Streamlit app externally on port 8080 is a common approach for quickly validating end-to-end functionality in development and demo scenarios — but it's not suitable for production. Free-tier ngrok URLs expire when the session ends, and traffic passes through third-party servers, making it inappropriate for handling sensitive data. The architecture documentation auto-generated by Deep Wiki includes data flow diagrams, typically described in text-based diagramming languages like Mermaid or PlantUML, which can be rendered as visual graphics directly on platforms like GitHub or Notion. These are a practical output format for AI agents to understand and document system structure.

From Autocomplete to a Real Engineer

This complete demo illustrates the fundamental difference between Devin and traditional code completion tools: a cloud agent that can self-test its work and send back a video, automation that can fix CI and triage bugs while you sleep, AI code review that can find and fix real bugs, deep security scanning that can identify injection attacks and patch them, and Outpost for connecting your own hardware.

Of course, the demo also surfaced situations requiring human involvement — GitHub authorization configuration, merge conflict resolution, and secret authorization method selection, for example. In all these cases, Devin proactively asked rather than deciding unilaterally. This is both a safety design choice and a sign that "fully unattended" operation still has a ways to go. For teams evaluating cloud-based coding agents, Devin offers a remarkably comprehensive capability profile.

(Note: This article is based on a hands-on demo from a single YouTube source, which was sponsored by Devin.)