Put an AI Reviewer on Every PR with OpenCode + Railway: A One-Hour Webhook Build

Code Review Is Becoming the New Bottleneck

When AI coding agents can submit Pull Requests far faster than any human can read them, a new problem emerges: review capacity becomes the true bottleneck in the development pipeline. We used to worry about writing code fast enough; now, agents can churn out PRs in minutes while engineers still have to manually read every diff line by line. This supply-demand imbalance slows down merge velocity and makes quality control increasingly difficult.

The solution proposed in the original article hits the nail on the head: if agents can write PRs, why not let another AI read them? By deploying a Webhook service on Railway, OpenCode runs in headless mode against each PR's diff, then posts its findings as review comments directly on the PR — the entire process is read-only and never touches any code.

The Core Design of This System

The architecture of this automated review system is straightforward and can be broken down into three key stages.

Webhook Trigger

The entry point is a Webhook endpoint deployed on Railway. Whenever a repository receives a new Pull Request (or a new commit is pushed to an existing PR), the platform sends an event notification to this endpoint. Railway, as a lightweight deployment platform, is well-suited for hosting small, event-driven, on-demand services like this — no need to manage your own servers.

A Webhook is an "event push" mechanism: when a code hosting platform (such as GitHub or GitLab) detects a specific event (like a PR being created or a new commit being pushed), it proactively sends an HTTP POST request to a pre-registered URL, with the request body containing structured event data (PR number, repository info, diff URL, etc.). This is the opposite of traditional "polling" — instead of your server asking the platform every few seconds "any new events?", the platform notifies you the moment something happens. Railway's role here is to host this Webhook receiver, providing a publicly accessible HTTPS endpoint. Railway's advantages include simple deployment, auto-scaling, and usage-based billing — making it extremely cost-efficient for low-frequency, short-lived event-driven services.

OpenCode Running Headless

Upon receiving a PR event, the service invokes OpenCode in headless mode — meaning no GUI or interactive terminal is required; it runs purely via command line or programmatic invocation. OpenCode reads the PR's diff, analyzes the changes, and identifies potential bugs, style issues, logic flaws, or security vulnerabilities. This is the intelligent core of the entire system, where AI plays the role of a tireless junior reviewer.

OpenCode is an open-source AI coding agent designed to support non-interactive invocation from the command line — which is the technical foundation of its "headless mode." Unlike editor-embedded completion tools such as GitHub Copilot, OpenCode is closer to a programmable automated task executor: it accepts instructions, reads code context, calls an underlying large language model (such as Claude or a GPT-series model), and outputs structured results. This programmability makes it ideal for embedding into CI/CD pipelines or Webhook services as an analysis module that requires no human intervention. In a PR review scenario, the input is typically the diff text retrieved via the GitHub API (the changes between two commits), which OpenCode passes as context to the model for analysis and reasoning.

Read-Only Comments — Never Touching the Code

A critical boundary this design enforces: the AI reviewer never modifies code. Its only output is review commentary, posted as review comments on the PR. This is a wise design choice — it keeps the decision to make changes firmly in human engineers' hands, with the AI responsible only for flagging and suggesting, eliminating the risk of an automated system making uncontrolled changes.

Why "Read-Only" Is the Key Trade-off

It's worth unpacking the engineering philosophy behind the "never touches the code" principle. In many AI-assisted coding workflows, the instinct is to let AI directly fix issues. But in the code review context, keeping AI read-only is actually the more mature approach.

The essence of review is providing a second set of eyes — helping humans catch things they might have missed, not replacing human judgment on merge decisions. If the AI writes, reviews, and modifies code all at once, it loses its value as an independent check. Staying read-only means every AI comment gets confirmed by a human, improving efficiency while preserving a necessary quality gate. It also lowers the psychological barrier to adoption: teams don't need to worry that an automated script might silently break production code.

What "One-Hour Build" Really Signals

The original article frames this as a "one-hour build," and that time budget itself sends an important message: adding AI to code review is no longer a large-scale project requiring a dedicated team and weeks of work.

With off-the-shelf AI coding tools like OpenCode and plug-and-play deployment platforms like Railway, individual developers or small teams can wire up an automated reviewer for their repositories in a single afternoon. This low barrier is changing how AI tools get adopted — shifting from "capabilities exclusive to big tech" to "workflow components anyone can reuse."

For teams already using AI agents to generate code at scale, adding a companion AI reviewer is almost an inevitable complement. Only when both production and review are accelerated by AI can the entire pipeline stay balanced — otherwise, review becomes the bottleneck.

Things to Watch Out For When Deploying

Despite the system's simplicity, there are a few real trade-offs to consider in practice.

First is review quality and reliability. AI reviewers will produce false positives and false negatives. Teams need to calibrate their expectations — it's an assistant, not an authority, and engineers must retain their own judgment. Second is cost control: calling a large language model to analyze diffs on every PR incurs ongoing API costs that scale with PR frequency and diff size, so consider truncating or chunking very large diffs. Finally, context limitations are a real constraint: a diff alone often doesn't capture the full context of a codebase, so AI-generated comments may lack the global perspective needed to be fully accurate.

These limitations don't negate the value of the approach — they're a reminder that the AI reviewer is best positioned as a first-pass filter: quickly catching obvious issues, formatting problems, and common anti-patterns, freeing up human reviewers to focus their attention where judgment matters most — architecture and business logic.

On the topic of context limitations: diffs provided by platforms like GitHub only include the added and removed lines from the current change, while many code review issues require understanding call chains, class inheritance, or cross-file business logic. Common mitigation strategies today include fetching the full content of relevant files alongside the diff and passing them to the model together (at the cost of significantly higher token usage), or using RAG (Retrieval-Augmented Generation) to supply the model with a structured index of the codebase. For large monorepos, the diff itself may exceed the model's context window limit, requiring chunking or restricting review to only the most heavily changed files. These engineering details are often where the biggest gap lies between a working demo and a production-ready system.

Final Thoughts

Deploying an AI reviewer on every Pull Request represents a concrete new paradigm: AI coding tools extending from "helping you write" to "helping you review." When code production is accelerated by agents, automating the review stage is a natural follow-on. The OpenCode + Railway combination demonstrates that this path is both viable and low-barrier — with minimal engineering investment, teams can equip their code pipeline with a tireless first line of defense.