Malware Triage Under Limited LLM Budgets: A Cross-Model Comparative Study

TRIAGE-CPG models LLM static triage as budget-constrained evidence acquisition, revealing that model choice is strategy choice.
TRIAGE-CPG formalizes LLM-assisted static binary triage as budget-constrained evidence acquisition: with samples averaging about 1,000 functions, models inspect only about 40 before trading off safe allowance, malicious flagging, and escalation. It builds a fixed three-stage pipeline based on code property graphs (CPGs) and evaluates eight closed-source and open-weight models on 1,000 Windows PE samples under a zero-shot protocol. The best model reaches 0.805 strict balanced accuracy and 0.855 tBA. The key finding is marked strategy divergence: high-specificity models have small, precise escalation queues, while recall-oriented models escalate more broadly, and both closed-source and open-source models fall along this spectrum. The core conclusion is that there is no universal deployment configuration—model choice itself determines the trade-off among allowance, flagging, and escalation, so security teams must select models according to their own risk preferences.
When security analysts face an unknown binary, they often need to decide quickly: is it safe to allow, should it be flagged as malicious, or does it need to be escalated for deeper analysis? Large language models (LLMs) can assist decision-making by interpreting decompiled code, but a single binary may contain hundreds or thousands of functions, making exhaustive LLM analysis impractical. A recent study called TRIAGE-CPG formalizes static triage as an "evidence acquisition under budget constraints" problem and systematically compares how multiple LLMs behave in this setting.

Core question: How do LLMs select evidence under a limited budget?
The research team points out that when static triage is framed as a budget-constrained evidence collection process, the key question is no longer "can the model identify malicious code?" but rather "which functions will the model inspect, and how does it trade off safe allowance, malicious detection, and escalation for analysis?" To enable controlled comparison, they implemented TRIAGE-CPG—a fixed pipeline built on code property graphs (CPGs) of decompiled binaries.
The pipeline has three stages: first, a non-LLM heuristic selects seed functions; then an agent expands the evidence profile under a fixed budget; finally, a report is generated that issues a safe, suspicious, or malicious verdict along with a behavior report. This design ensures that different models are compared under identical conditions, eliminating interference from pipeline differences.
Code Property Graph (CPG) is a graph structure for representing program semantics, first proposed by the Joern project. It merges the abstract syntax tree (AST), control flow graph (CFG), and program dependence graph (PDG) into a single multi-edge graph: nodes represent program elements such as variables, expressions, and statements, while edges encode relationships such as syntactic containment, control-flow jumps, and data dependence. For decompiled binary code, a CPG can recover structure close to source-level information, so the LLM does not have to face obscure assembly instructions directly but instead reasons over an intermediate representation with types and dependency relationships. TRIAGE-CPG chose CPG as the LLM's input carrier precisely because it preserves semantics while significantly compressing the complexity of the raw binary.
Experimental design: eight models, one thousand samples, zero-shot protocol
The study evaluated eight closed-source and open-weight models on a dataset of 1,000 Windows PE binaries, with 600 samples used as the test set, and all models used the same zero-shot protocol. Two evaluation metrics were used: a strict binary interpretation (only samples flagged as malicious count as positive) and triage-balanced accuracy (tBA), which counts malicious samples that are either flagged or escalated as correct, while safe samples count as correct only when they are allowed.
Each binary contains on average about 1,000 functions, and models made their judgments after analyzing only about 40 of them. This "budget" constraint simulates real-world scenarios where analyst time is limited.
Zero-shot protocol means that during evaluation, the model receives no example demonstrations or fine-tuning specific to the task and relies solely on capabilities acquired during pretraining. In the malware triage setting, adopting a zero-shot protocol means that all eight models face exactly the same prompt template and function-selection process, eliminating interference from prompt-engineering differences and task-specific training data. The advantage is a fair comparison of the models' native reasoning abilities, but it also means the results reflect the models' "lower bound" performance—after targeted fine-tuning or few-shot prompting, both accuracy and strategy distributions may change significantly. When interpreting the 0.805 strict balanced accuracy, readers should understand it as a capability baseline under current zero-shot conditions.
Key finding: Model strategies diverge markedly
The best-performing model reached 0.805 strict balanced accuracy and 0.855 tBA. Specifically, this model flagged 65% of malware, escalated another 12%, and allowed 94% of safe samples. This result shows that LLMs already possess considerable capability on static triage tasks, but strategy differences between models are large.
The study divides models into two categories: high-specificity models produce small, precise escalation queues with a high proportion of malware; recall-oriented models escalate or flag samples more broadly. Notably, both closed-source and open-weight models are distributed across this spectrum, and neither category has a natural advantage.
Triage-balanced accuracy (tBA) is a composite metric designed in this study to align with real triage workflows. Traditional binary classification only cares about two outputs, "malicious/safe," but real analysts have a third action: escalate to deeper analysis. The scoring rule for tBA is: malicious samples count as correct if they are either flagged as malicious or escalated; safe samples count as correct only if they are explicitly allowed. This design encourages models to choose escalation rather than forcing a binary choice when uncertain, but it also means that over-escalating malicious samples is not penalized, while over-escalating safe samples lowers the score. Therefore, the gap between tBA and strict binary accuracy actually reflects the extent to which a model converts uncertainty into escalation actions.
No universal configuration: Model choice is strategy choice
The study's core conclusion is that there is no universal deployment configuration. Under limited static evidence, model choice itself determines the trade-off among allowance, flagging, and escalation. This means security teams need to select models based on their own business requirements (for example, whether they fear false negatives or false positives more), rather than expecting an "all-purpose" model.
This finding has direct implications for practical deployment: if a security operations center wants to reduce analyst workload, it can choose a high-specificity model to make the escalation queue more precise; if it cares more about coverage, it must accept more false positives and escalation volume. The study also cautions that LLM-assisted triage is not a replacement for human analysts but rather an evidence-screening tool under budget constraints.
Implications for security practice
The TRIAGE-CPG study provides a reproducible evaluation framework for applying LLMs to malware analysis. Its value lies not only in showing the capability boundaries of current models but also in revealing the strategy-selection issues behind differences in model behavior. For organizations that want to introduce LLMs to assist static triage, clarifying their own risk preferences and operational constraints and then evaluating candidate models accordingly may be more pragmatic than blindly pursuing the "strongest model."
The study also points to future directions: how to design better evidence-selection strategies, how to incorporate dynamic analysis signals, and how to enable models to achieve a better automatic balance between budget and accuracy are all questions worth exploring.
One-line Summary
TRIAGE-CPG models LLM static triage as budget-constrained evidence acquisition, revealing that model choice is strategy choice.
Paragraph Summary
TRIAGE-CPG formalizes LLM-assisted static binary triage as budget-constrained evidence acquisition: faced with samples averaging about 1,000 functions, a model inspects only about 40 functions before having to trade off safe allowance, malicious flagging, and escalation for analysis. The study builds a fixed three-stage pipeline based on code property graphs (CPGs) and evaluates eight closed-source and open-weight models on 1,000 Windows PE samples under a zero-shot protocol. The best model achieves 0.805 strict balanced accuracy and 0.855 tBA. The key finding is marked strategy divergence: high-specificity models have small, precise escalation queues, while recall-oriented models escalate more broadly, and both closed-source and open-source models are distributed along this spectrum. The core conclusion is that there is no universal deployment configuration; model choice itself determines the trade-off among allowance, flagging, and escalation, so security teams must select models according to their own risk preferences.
Related articles

Abnormal Error Rate Spike in Claude Mythos 5.2 and Fable 5.2
Claude Mythos 5.2 and Claude Fable 5.2 show an abnormal spike in request error rates, marked as investigating on the official status page. This article covers the incident overview, practical impact on developers, and recommended responses.

ReAct Runtime Context Trimming: Why Claude Code Acts at 70%
A deep dive into ReAct runtime context trimming: why Claude Code compresses at ~70% usage, why tool calls dominate, why messages must be dropped in pairs, and why codebases don't need RAG.

AI Agent vs LLM: How Cursor and Claude Code Actually Work
An in-depth analysis of the essential difference between AI Agents and LLMs: LLMs generate code suggestions, while Agents modify files and run tests via tool calls. Using a Spring Boot bug fix as an example, it breaks down the workflows, permission control, and verification mechanisms of Cursor and Claude Code.