#web scraping automation
174 related articles
·2 min
BrowserOS neo: The Open-Source Tool That Lets AI Agents Control Your Browser
BrowserOS neo is an open-source browser built for AI agents, running locally with your existing credentials. Lets Claude Code, Codex and other AI assistants complete real web tasks on your behalf.
Read more →
·3 min
Kitesurf: A V8 Isolate-Based Lightweight Browser Designed for AI Agents
Deep dive into Kitesurf—a lightweight browser built on V8 Isolates for AI Agents. Learn how its millisecond cold starts, high concurrency, and sandbox isolation solve traditional browser bottlenecks in AI automation.
Read more →
·3 min
Eliminating Fake Torrents in Sonarr: Building a Truly Hands-Off Media Library
In-depth analysis of why Sonarr grabs .exe/.scr fake torrents, with practical solutions including size minimums, Release Profile filtering, Prowlarr pre-filtering, and the case for private Trackers vs public indexers.
Read more →
·3 min
Cursor Browser Worker Parallelization: Practical Strategies for Working Within Rate Limits
Learn how to parallelize Cursor browser Workers from serial to parallel execution using distributed Worker pools, proxy pools, token bucket algorithms, and exponential backoff to compress 2000-3000 page scraping tasks from hours to 15-20 minutes.
Read more →
·3 min
Perplexity Comet's Declining Agent Capabilities: Why This AI Browser Is Becoming Timid
Perplexity Comet users report declining AI agent capabilities, with form-filling and automation tasks frequently refused. We analyze the causes from anti-automation detection, compliance risks, and model policy tightening perspectives.
Read more →
·3 min
Building a Local AI Agent Architecture: From Scattered Scripts to Your Personal Jarvis
A practical guide to consolidating scattered automation scripts into a local AI Agent hub. Covers Function Calling, Ollama+Qwen2.5 deployment, tool orchestration architecture, and a complete implementation roadmap.
Read more →
·4 min
SoulSync + slskd + Navidrome Self-Hosted Music Setup: Looks Great but Confusing in Practice
A detailed look at SoulSync, slskd, and Navidrome self-hosted music setup: component roles, file flow logic, and common pitfalls including Wishlist confusion and Spotify rate limiting.
Read more →
·3 min
Building an AI Agent from Scratch: A Full Development Process Retrospective and Pitfall Guide
A complete guide to building AI Agents from scratch based on real developer experiences: task selection, tool comparison (no-code vs frameworks vs hand-written), stability challenges, and evaluation criteria.
Read more →
·3 min
Perplexity Computer Real-World Test: 15,000 Credits Burned with Zero Results
A user burned over 14,000 Perplexity Computer credits building an AI Agent workflow with zero output. We break down the three fatal flaws and the Claude+OpenAI workaround.
Read more →
·2 min
AgentQuartz: Real-Time Claude and Cursor Usage Monitoring from Your macOS Menu Bar
AgentQuartz is a macOS menu bar app for real-time monitoring of Claude and Cursor AI coding usage quotas. Local-first architecture protects privacy, core features are free forever, Pro is a one-time $3 purchase.
Read more →
·2 min
Criblist: Find San Francisco Apartments Like Swiping on Tinder
Criblist is a San Francisco rental aggregation tool that brings Tinder-style swiping to apartment hunting, integrating real-time listings from Craigslist and local property managers with budget, neighborhood, and must-have filters.
Read more →
·4 min
Banquish: A Mac Productivity Tool That Pins Live Web Fragments onto a Canvas
Banquish is a Mac app that clips live web fragments onto a free-form canvas, eliminating tab-switching hell. Combined with AI Agent automation, it creates personalized information dashboards.
Read more →
·3 min
ChatGPT Autonomous Ticket Hunting in Action: AI Agent Auto-Monitoring and Ticket Purchase Demo
Demo of ChatGPT Work as an AI Agent autonomously browsing AMC's website for IMAX tickets, setting hourly auto-checks for cancellations, and sending notifications.
Read more →
·4 min
WebMCP in Practice: How MakeMyTrip Is Reshaping the Travel Booking Experience
India's largest OTA platform MakeMyTrip uses WebMCP to standardize AI Agent interactions with web apps, replacing fragile DOM scraping with natural language-driven test automation and simplified complex booking scenarios.
Read more →
·4 min
WebMCP in Practice: How MakeMyTrip Is Reshaping the Travel Booking Experience
India's largest OTA platform MakeMyTrip uses WebMCP to standardize AI Agent interaction with web apps, solving DOM scraping fragility, enabling natural language test automation, and simplifying complex international flight bookings.
Read more →
·3 min
MediaCrawler: Deep Dive into an Open-Source Crawler Framework Covering Seven Major Platforms
Deep analysis of MediaCrawler, a popular GitHub open-source project using Playwright browser automation for multi-platform crawling across Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, and Zhihu.
Read more →
·3 min
Cloudflare Blocks Anti-Scraping Proxies: IP Reputation Is the Real Culprit
Analysis of why FlareSolverr and Byparr fail against Cloudflare. The real issue isn't CAPTCHA solving — it's IP reputation. Learn about VPN pitfalls, static IP risks, and practical solutions.
Read more →
·3 min
MediaCrawler: Deep Dive into an Open-Source Crawler Framework Covering Seven Major Platforms
Deep dive into the popular GitHub project MediaCrawler — a Playwright-based multi-platform crawler framework supporting data collection from Xiaohongshu, Douyin, Kuaishou, Bilibili, Weibo, Tieba, and Zhihu.
Read more →
·3 min
GitHub Daily · July 27: AI Agent Skill Ecosystem Explodes, Claude's Capability Boundaries Keep Expanding
GitHub Trending July 27: AI Agent Skills explode as claude-video, impeccable, and last30days-skill extend model capabilities without modifying models themselves.
Read more →
·3 min
Open-Source Automated News Briefing Generator: A Zero-Technical-Barrier AI Deployment Analysis
An open-source automated news briefing system. No coding needed—just let an AI read the project docs to complete the entire deployment. Six-stage pipeline, four-channel search covering 16+ platforms, smart classification and dedup, daily auto-push to Feishu, completely free.
Read more →