Building an Agent on AWS: Score GitHub Repos Before You Clone

Title

Building an Agent on AWS: Score GitHub Repos Before You Clone

Meta Description

FakeGit campaign involved 7,600 malicious GitHub repos and 14 million downloads, and agents actively recommend malicious repos. A developer built RepoPilot with AWS Lambda, Amazon Nova Pro, and Strands to give any GitHub repo a 0-100 trust score and flag dangerous install steps before you clone.

Article Content

When your AI coding assistant tells you to clone a repo and run an install command, you probably just do it. In 2026, that habit has become dangerous. The FakeGit campaign uncovered by Island Security Research involves roughly 7,600 malicious GitHub repos, 6,600 fake accounts, and over 14 million malicious downloads—more than 800 of them disguised as AI skills or MCP servers, exactly the kind of components we're now wiring into our agents.

To address this, developer Sarvar Nadaf built RepoPilot: an AWS agent that points at any public GitHub repo, returns a 0-100 trust score in a minute or two, and flags dangerous install steps. It can't replace a manual audit, but it bridges the gap between "a bot told me to run it" and "I actually ran it."

The 2026 Threat Has a New Entry Point

According to Island Security Research, more than 800 repos in the FakeGit campaign were disguised as AI skills and MCP servers. Researchers named another class of attack AgentBaiting: in their tests, Claude Code, Gemini, and ChatGPT each independently recommended malicious repos—without anyone giving them a link. Ask an agent for a "free Walmart MCP server" and it might return the attacker's repo as its top pick, because the malicious README was read as normal documentation. The lure is no longer just a convincing README; the lure is your agent itself—endorsing a repo it never checked.

Two other incidents corroborate this theme. As reported by The Hacker News, the GhostAction campaign discovered by GitGuardian affected about 1,900 GitHub accounts and over 30,000 repos, with a payload that attempted to steal 3,174 secrets from CI environments, including npm, PyPI, DockerHub, and AI vendor credentials. As reported by Infosecurity Magazine, the SANDWORM_MODE worm discovered by Socket wrote malicious MCP servers into the local configs of Claude Desktop, Cursor, VS Code Continue, and Windsurf, and harvested API keys from nine LLM vendors.

The common thread across these three incidents: the dangerous part is rarely the code you read—it's the install step you execute on trust: curl ... | bash, postinstall hooks, "download this ZIP and run it." And now, the step that recommends it is often the agent itself.

MCP (Model Context Protocol) is a protocol proposed and open-sourced by Anthropic in late 2024 to let AI models connect to external tools, data sources, and services in a standardized way. An MCP server is essentially a set of capability interfaces exposed to an agent, which can call them on demand to read/write files, query databases, or execute commands. Because MCP servers are typically distributed as "install and use," attackers can hide malicious logic in the server's install scripts or dependencies, which then execute within the agent's permission boundary once connected. The 800+ repos disguised as MCP servers in the FakeGit campaign exploit this trust model: developers tend to think "it's just adding a tool" and don't audit it like business code.

AgentBaiting is a new attack paradigm proposed by Island Security researchers. Instead of directly tricking humans into clicking a link, attackers craft repo content highly attractive to agents (e.g., READMEs full of keywords, examples, and friendly explanations), so that when an agent answers a user's natural-language request, it actively retrieves and recommends that repo. Because users often view an agent's recommendation as "filtered by the model," this endorsement effect is more deceptive than traditional phishing. Defending against AgentBaiting can't rely solely on the model's own judgment; it requires inserting an independent, explainable check layer between the agent's recommendation and the user's execution.

How RepoPilot Scores Repos

Enter owner/repo and it returns a credit-score-like 0-100 rating composed of five weighted dimensions, each traceable to signals in repo metadata and files—not gut feeling:

Dimension	Weight	What It Reads
Security	30%	Malicious patterns in install steps, e.g., curl | bash
Trustworthiness	25%	Star-to-repo-age ratio, fork and contributor signals, star-farming patterns
Maintenance	20%	Recent commits, issue open/close ratio, release cadence
Maturity	15%	Repo age, contributor count, overall footprint
Documentation Quality	10%	README depth, license, examples

Two dimensions carry most of the weight against 2026 threats. The security dimension scans install steps for patterns of hidden malware; any hit is a critical issue, and a critical issue caps the total score outright—if the installer is designed to steal your keys, "it runs" means nothing. The trustworthiness dimension targets the star-farming patterns FakeGit relies on: the campaign uses one-character-off usernames and a handful of stars to make repos look like established projects at first glance. Repos whose reputation doesn't match their age are called out in plain language.

Enter a username instead of a repo path, and it analyzes the developer profile: footprint, primary languages, top repos by stars. Those 6,600 fake accounts are the other half of the attack—auditing code isn't enough; you have to audit the people publishing it.

An Honest Example

Run ollama/ollama through it, and RepoPilot gives it a D. Ollama is a legitimate project, so this isn't a malice verdict. What triggers the flag is its official install command curl -fsSL https://ollama.com/install.sh | sh—piping a remote script straight into your shell. RepoPilot doesn't invent a threat; it says: read this before running. Even trusted repos have install steps worth a second look, and an honest tool admits that.

Star farming is a common reputation-forging technique in open-source supply chain attacks. Attackers use bulk account registration, automated scripts, or purchased services to rapidly accumulate stars for malicious repos, making them appear more credible in search results and agent recommendations. The FakeGit campaign also combined a "typosquatting" strategy: registering usernames or repo names that differ from well-known projects by only one or two characters, paired with a handful of stars and a plausible README, so the repo visually resembles an established project. The challenge in detecting star farming is that crude patterns (e.g., a new repo gaining many stars in a short time) are easy to spot, but patient attackers can evade ratio-based rules through long-term account nurturing, mutual starring, and gradual growth. RepoPilot's trustworthiness dimension attempts to capture this mismatch using cross-features like star-to-repo-age ratio and fork and contributor signals.

Architecture: Python Scores, the Model Just Tells the Story

The tech stack is deliberately lean: Amazon Nova Pro on Amazon Bedrock handles reasoning and narration, AWS Lambda (python3.12) hosts the agent, Amazon API Gateway fronts it for the web UI, a local CLI is built on the Strands Agents SDK, and seven pure-Python tools feed the scoring engine.

The most critical design trade-off: the 0-100 score, critical flags, and version detection are all computed deterministically in Python, unit-testable without touching AWS; the LLM only narrates the facts produced by the tools and never touches the score. This makes the score provably correct, and the model can't describe something unsafe as safe.

The author offers two lessons for other developers building Bedrock agents on Lambda. First, verify model access before writing agent code—a single converse call tells you in half a second whether Nova Pro is reachable, far cheaper than building the whole project and then debugging AccessDenied. The reason for choosing Nova Pro over Claude: this task is fundamentally narration rather than reasoning, the score is already decided by Python, and Nova Pro is cheaper per token and fast enough for narration. Second, cut heavy dependencies—bundling requests into the Lambda zip hits a wall; its bundled urllib3 imports something removed in Python 3.12. Rewriting the GitHub fetcher with the standard library shrank the package from 400KB to 13KB and eliminated the errors.

The dangerous-pattern scanner also went through a revision. The first version flagged Flask as unsafe because its docs mention 127.0.0.1; the author taught the scanner to ignore localhost and private address ranges while still catching public-IP installers. The cost of a false positive on a trusted framework is higher than missing an edge case.

The cost model changes accordingly: scoring is pure Python, narration is one short Nova Pro call, scanning on your own Lambda is cheap, with no per-seat subscription and no third-party API billing per repo. The project has 25 tests, including 200 random repos, to prove the weighted formula doesn't produce contradictory results and that critical flags always cap the grade.

Amazon Nova Pro is one of the multimodal foundation models AWS offers on Amazon Bedrock, positioned as a general-purpose reasoning model balancing cost and performance. In RepoPilot's architecture, Nova Pro doesn't participate in score calculation; it's responsible for transcribing the structured facts produced by Python tools (scores, critical flags, version info) into human-readable narration. This division of "deterministic computation + model narration" is a key security design: the score and capping logic are entirely determined by unit-testable Python code, and the model cannot, through hallucination or prompt injection, describe an unsafe repo as safe. The choice of Nova Pro over Claude is based on this too—the task is fundamentally narration rather than complex reasoning, and Nova Pro has advantages in per-token cost and latency.

Test Results and Known Blind Spots

For the Flask repo, the deployed Lambda fetched 236 files and gave it 98/100 (A grade): security, trustworthiness, maintenance, and maturity all 100, documentation 85; it identified the Python stack and read python >=3.11 from the actual pyproject.toml. langchain-ai/langchain got a clean 100, ollama/ollama dropped to a D due to its curl | sh install method, and @torvalds resolved to Linus Torvalds, whose repos total about 263,000 stars.

The author also lists RepoPilot's blind spots. It reads the surface, not the payload: it can flag suspicious install steps but won't unzip a ZIP or reverse-engineer the binaries inside—which is exactly where FakeGit hides its real payload. So a clean score means "no obvious red flags," not "audited and safe." A high score also isn't a health certificate: scoring rewards maintenance, stars, and documentation, and a well-run, popular repo that turns malicious in some new release may keep a high score for a while—reputation always lags reality. Stars and age can be gamed over the long term; star-farming detection catches crude patterns like "a three-week-old repo with 40,000 stars," but patient attackers can evade it by nurturing accounts.

The difference between surface scanning and payload analysis is key to understanding RepoPilot's capability boundary. Surface scanning reads text files, configs, and install commands in a repo, and can identify patterns like curl | bash, postinstall hooks, and suspicious download links; but it won't unzip ZIPs, decompile binaries, or dynamically execute install scripts. The real malicious payload in the FakeGit campaign is often hidden in executables inside archives or obfuscated scripts, which can only be found by downloading and analyzing those artifacts. Therefore RepoPilot's clean score should be understood as "no obvious red flags found," not "audited and safe." For code about to be granted real access, you still need to manually read the install steps and key files.

Why Open Source

A closed-source hosted "repo checker" is wrong for this problem, and the reasons compound: your scans stay private—which repos and usernames you're suspicious of is itself a signal—and RepoPilot runs on your own AWS account, or entirely via the local CLI, sending nothing to third parties. The cost model follows: open weights plus your own Lambda means you can screen every repo an agent suggests without per-call fees. When new attack patterns emerge this month, you add them directly to the pure-Python rules instead of waiting for a vendor roadmap. There's no lock-in either: the narration layer can be swapped for any supported model, and the scoring engine doesn't care.

When the threat comes from the open-source supply chain, the defense should be open too, so anyone can read exactly how the score is computed and then decide whether to trust it.

RepoPilot solves a thirty-second gut check, catching the obvious and lazy portion of attacks—which covers most samples in this campaign. For code about to be given real access, you still need to read it yourself.

One-line Summary

RepoPilot uses deterministic Python scoring plus LLM narration to give GitHub repos a 0-100 trust score, defending against supply chain attacks in the agent era.

Paragraph Summary

The article uses the 2026 FakeGit campaign as background to reveal that AI coding assistants and agents are becoming new entry points for supply chain attacks: about 7,600 malicious repos, 6,600 fake accounts, 800+ disguised as AI skills or MCP servers, and AgentBaiting causing Claude Code, Gemini, and ChatGPT to independently recommend malicious repos. Developer Sarvar Nadaf built RepoPilot, which gives a 0-100 trust score across five weighted dimensions (security 30%, trustworthiness 25%, maintenance 20%, maturity 15%, documentation 10%), with scores computed deterministically in pure Python and the LLM only narrating, ensuring the model can't confuse safety judgments. The tool can flag dangerous install steps and identify star-farming patterns, but it also explicitly acknowledges its blind spots: it reads the surface without unpacking payloads, and a high score isn't a security audit.