Browser-Use WebUI Installation & Configuration Guide: AI Browser Automation in Practice

Introduction

The AI Agent track continues to heat up in 2025, and browser automation has become an indispensable capability within the Agent ecosystem. Whether it's auto-filling forms, scraping data, or executing a chain of complex web operations, AI-driven browser automation tools are redefining how we interact with the internet.

An AI Agent is an AI system capable of perceiving its environment, making autonomous decisions, and executing actions to achieve goals. Unlike traditional conversational AI, Agents possess "agency" — they can not only generate text responses but also invoke tools, operate software, and interact with external systems. In 2025, leading companies like OpenAI, Google, and Anthropic have all made Agent capabilities a core strategic direction, and the ecosystem of Agent frameworks (such as LangChain, AutoGPT, CrewAI) is maturing rapidly. Browser automation is one of the most practically valuable directions within Agent capabilities, because the internet hosts the vast majority of human workflows — from information retrieval and data entry to online transactions, nearly all business processes can be completed through a browser.

The focus of this article, Browser-Use WebUI, is a visual interface version built on top of the popular open-source GitHub project Browser-Use. It transforms browser automation — which previously required writing code — into something as simple as "typing a chat message." Describe what you want to do in natural language, and the AI will automatically complete it in the browser for you.

What Is Browser-Use WebUI

Browser-Use is an open-source project that has earned a large number of Stars on GitHub. Its core capability is enabling AI models to directly control browsers to execute various tasks. The official repository showcases several impressive use cases:

• Automated Job Applications: Read resume content, search for machine learning positions, save results, and automatically submit applications
• Flight Searches: Automatically search for flights to a target city on travel websites
• Data Collection: Filter models by license type on HuggingFace, sort by popularity, and save the top five

From a technical architecture perspective, Browser-Use's core is built on the combination of Large Language Models (LLMs) and the Playwright browser automation engine. Its working principle can be summarized as a "perceive-decide-execute" loop: First, the system uses Playwright to capture the current webpage's DOM structure and interactive elements, passing this information (sometimes including page screenshots) as context to the LLM. Then, the LLM determines what action to take next based on the user's natural language instruction and current page state (such as clicking a button, typing text in an input field, scrolling the page, etc.). Finally, the system translates the LLM's decision into specific Playwright API calls and executes the corresponding action in the browser. This loop continues until the task is completed or the maximum step limit is reached.

Browser-Use WebUI wraps a clean graphical interface on top of this, supporting integration with multiple AI models — including Gemini (free with vision capabilities) and DeepSeek (cost-effective) — lowering the barrier to entry significantly.

Four Practical Scenario Demonstrations

Scenario 1: Automatically Filling Web Forms

The first task is having the AI automatically open a Waitlist registration page and fill in user information. This page was built with Rapidata Agent, with two test data entries already in the backend.

The actual operation went very smoothly: After clicking "Run Agent," the browser automatically opened the target page. The AI marked all interactive elements on the page with differently colored boxes, then sequentially filled in the name, email, and other fields, and finally automatically clicked the "Join Waitlist" button. Refreshing the backend confirmed that the newly submitted data had been successfully stored.

It's worth elaborating on the technical details of how the AI marks interactive elements. The system performs DOM parsing on the current page, extracting all interactive HTML elements (such as buttons, links, input fields, dropdown menus, etc.), assigns a unique number to each element, and visually marks them with colored borders on the page. This annotation information is organized into structured text descriptions (e.g., "[1] Input: Name, [2] Input: Email, [3] Button: Submit") and sent to the LLM along with the page screenshot. The LLM only needs to respond with instructions like "click element [3]" or "type 'John Doe' in element [1]," and the system can precisely locate and operate the corresponding page elements. This design cleverly transforms complex web operations into text reasoning problems that LLMs excel at handling.

This scenario fully demonstrates Browser-Use's strength in form automation — it's perfectly suited for handling repetitive tasks like batch registrations and data entry.

Scenario 2: Automatic Web Content Summarization

The second task is having the AI open Sam Altman's blog and summarize the key points of the latest article.

The AI accurately located Sam's latest blog post published that day and color-coded different sections of the article. Terminal logs showed that the AI captured the complete blog content (which was quite lengthy) and ultimately produced a clearly structured summary of key points.

This scenario demonstrates Browser-Use's value in information extraction and content comprehension — useful for daily news monitoring and quick reading scenarios.

Scenario 3: Video Search and Playback

The third test had the AI search for a specific user ID on Bilibili (B站), click the first video in the search results, and play it. The AI successfully completed the search and location, finding the target video and clicking the first result. However, since the current browser environment didn't support the H5 player, the video failed to play — revealing the limitations of automation tools in certain runtime environments.

This limitation is related to the underlying Playwright engine. Playwright is a modern browser automation framework developed and open-sourced by Microsoft, supporting three major browser engines: Chromium, Firefox, and WebKit. Compared to the earlier Selenium, Playwright offers significant advantages in speed, stability, and modern web feature support — it natively supports auto-wait mechanisms (no need to manually set sleep), multi-tab and iframe operations, network request interception and mocking, and built-in screenshot and screen recording capabilities. However, browser instances launched by Playwright typically run in headless or restricted mode, and certain features that depend on specific browser plugins or hardware acceleration (such as H5 video players) may not work properly.

Scenario 4: E-commerce Data Collection (Model Comparison)

The final task was more challenging: open Amazon, search for Bluetooth headphones, and scrape the titles and prices of the top five products in the search results.

Here, a noteworthy difference in model performance emerged:

• DeepSeek Model: Failed with errors when executing this task
• Gemini Model: Was able to continuously progress through the operations, with its ongoing interactions visible in the browser

There are deep technical reasons behind this difference. Gemini is Google's multimodal large language model series, with its core advantage being native visual understanding — it can not only process text but also directly "see and understand" webpage screenshots. This is particularly important for browser automation scenarios because some page elements' semantics are difficult to accurately determine from DOM structure alone, and visual information provides crucial supplementary context. DeepSeek is a cost-effective model from China's DeepSeek company, with API pricing far below commercial models like GPT-4. It excels in reasoning ability but is relatively weaker than Gemini in multimodal (especially visual) capabilities. Amazon's product search results pages have complex layouts with numerous ad slots, recommendation cards, and dynamically loaded content. Visual capability provides an additional information dimension when understanding visually dense elements like product cards and price tags, which explains why Gemini performed better on this task.

This demonstrates that different AI models genuinely have varying capabilities when handling complex web interaction tasks. Choosing the right model based on task characteristics directly impacts automation success rates.

Model and Parameter Configuration Details

Gemini Model Configuration

Browser-Use WebUI supports multiple Agent Types. One thing to note: if you select the "Original" type, you may get a prompt about a missing JSON file (related to Google's authentication mechanism). If you encounter this, it's recommended to switch the Agent Type to Custom to resolve the issue.

Another common pitfall: when using Edge browser, if you check the "Use your own browser" option, errors may occur. The fix is simple — uncheck that option and let the tool use its built-in browser instance.

DeepSeek Model Configuration

DeepSeek's configuration steps are straightforward:

1. Set Agent Type to "Original"
2. Select DeepSeek as the model
3. Keep the Base URL at the default Endpoint address
4. Enter the API Key in the .env file in the project root directory

Regarding the .env file: The project comes with a .env.example sample file. Simply remove .example from the filename and fill in your own API Key. The .env file is a widely used environment variable configuration method in software development that separates sensitive information (such as API keys, database passwords, etc.) from code, preventing keys from being hardcoded in source code or accidentally committed to version control systems.

Complete Installation and Deployment Steps

Here's the complete process for installing Browser-Use WebUI using a Conda environment:

Step 1: Create a Python Virtual Environment

conda create -n browser-use python=3.12
conda activate browser-use

The official documentation recommends Python 3.11, but testing confirms that 3.12 also works fine. Conda is an open-source package management and environment management system, originally developed for Python and now supporting multiple languages. The benefit of using Conda to create virtual environments is isolating independent Python versions and dependencies for each project, avoiding dependency conflicts between different projects.

Step 2: Clone the Repository and Install Dependencies

git clone <repository-url>
cd browser-use-webui
pip install -r requirements.txt

Step 3: Install Playwright Browser Drivers

playwright install

Don't skip this step! Many people skip it and go straight to launching the WebUI, only to encounter errors. Playwright is the underlying browser automation engine and requires separate installation of the corresponding browser drivers to function properly. After executing the playwright install command, the system automatically downloads binary files for three browsers — Chromium, Firefox, and WebKit (totaling several hundred MB). These are the runtime components that Playwright needs to launch and control browsers in the background, and they are independent of any Chrome or Firefox browsers already installed on your computer.

Step 4: Configure API Key Environment Variables

cp .env.example .env
# Open the .env file with an editor and fill in your API Key

Step 5: Launch the WebUI Service

After completing the above steps, launch the WebUI and access the corresponding address in your browser to start using it.

Summary and Outlook

Browser-Use WebUI is a highly practical AI browser automation tool, with core advantages in three areas:

1. Low barrier to entry: The graphical interface makes it easy for non-coders to use
2. Flexible model selection: Supports multiple models including Gemini and DeepSeek, allowing free combination based on task complexity and budget
3. Wide range of applications: From form filling and content summarization to e-commerce data collection, it covers a large number of daily automation needs

2025 is widely considered the inaugural year of Agents, and browser automation — whether Browser-Use or Claude's Computer Use — is a critical piece in the Agent capability landscape. It's worth noting that Claude Computer Use is a computer control capability launched by Anthropic in late 2024, which allows the Claude model to directly control the entire desktop environment (not limited to browsers) by recognizing screen content through screenshots and simulating mouse and keyboard operations. Unlike Browser-Use's precise control based on DOM parsing, Computer Use is closer to how humans operate — deciding where to click by "looking" at the screen. Each approach has its pros and cons: DOM parsing is more precise and faster but depends on the web page structure being parseable; visual control is more universal and can theoretically operate any graphical interface application, but with relatively lower accuracy and speed. Currently, the industry also has similar projects like Microsoft's UFO and Google's Project Mariner, and browser and computer automation is becoming one of the most fiercely competitive tracks in the AI Agent space.

Mastering these tools early can help us build more complex automated workflows and truly unleash AI's productivity potential.