System One Decision Models Applied to Malware Detection for the First Time: An Analysis of the S1Mal Ensemble Approach

Systematically evaluates System One decision models in PE malware detection and proposes the S1Mal ensemble approach.
The study provides the first systematic evaluation of System One decision models in Windows PE malware detection. These models aim to simultaneously provide calibratable probability scores and readable atomic decision bases. In an isolated environment, experiments used Ghidra and capa to convert binaries into textual states, with three readout methods: A0 (single judgment question), A1 (32 atomic behavior questions + gradient boosting tree aggregation), and A2 (state embedding classification head), covering CLM-8B, ten open decision models, and two LLMs read via logits. Results show that A0's AUROC spans only 0.4025-0.8590, with substantial static evidence wasted; A1 improves each model's AUROC by an average of 54.61%; A2 improves CLM-8B by 0.2749 to 0.9943. S1Mal integrates A1 atomic answers with A2 scores, achieving 0.9946 AUROC, detecting 93.47% at 1.45% FPR and 59.15% at 0.1% FPR (46.59% with A2 alone).
Malware detection has long faced a contradiction: feature-based detectors can produce scores calibrated at a fixed false positive rate (FPR), but the basis for their decisions is difficult for analysts to read directly; while large language models (LLMs) based on decompiled code can generate readable textual explanations, their confidence is uncalibrated. System One decision models attempt to bridge this gap—they answer a typed question about a text, producing a probability distribution over claimed answers in a single forward pass. A recent study systematically evaluated how such models perform in Windows PE malware detection for the first time, and proposed an ensemble approach called S1Mal.

Problem Background: The Split Between Calibrated Scores and Readable Decisions
Traditional feature-based detectors can provide well-calibrated scores at a fixed FPR, which is a fundamental requirement for security operations—analysts need to know "what false positive risk this score corresponds to." But the decision logic of such detectors is often a set of feature weights that are hard to interpret.
LLMs take a different path: generating text on decompiled code so analysts can understand what the model is "thinking." The problem is that the confidence of this generated text is uncalibrated and cannot be directly translated into actionable thresholds. The design goal of System One decision models is precisely to satisfy both needs at once: output probabilities over claimed answers (calibratable), and make every decision traceable to atomic question answers (readable).
Calibration refers to the consistency between a model's output confidence scores and its actual accuracy. A well-calibrated model, when it gives a malicious probability of 0.9, means that among all samples given that score, approximately 90% are indeed malicious. For security operations, calibration is a prerequisite for threshold setting: analysts need to map scores to specific false positive risks (e.g., "a score above 0.8 corresponds to a false positive rate below 0.1%"). Many deep learning models (especially LLMs) tend to output overconfident or underconfident probabilities, and uncalibrated scores cannot be directly used for operational decisions; post-processing methods such as Platt scaling, isotonic regression, or temperature scaling are usually needed for calibration.
AUROC (Area Under the ROC Curve) is a metric measuring the overall discriminative ability of a binary classifier, ranging from 0.5 (random) to 1.0 (perfect); it measures the probability that the model ranks a random positive sample ahead of a random negative sample. AUROC is insensitive to class imbalance, so it is widely used in scenarios like malware detection where the ratio of positive to negative samples is highly skewed. But AUROC has limitations: it aggregates performance across all thresholds and cannot directly reflect the detection rate at a fixed low FPR (e.g., 0.1%), which is precisely the metric security operations care about most. Therefore, the study also reports detection rates at specific FPRs.
Experimental Design: Isolated Environment and Three-Level Readout
The study built a complete analysis pipeline in an isolated virtual machine: Ghidra and capa are responsible for decomposing each binary file, and the host renders the records into a standardized textual state—covering summaries of file headers, import tables, MITRE ATT&CK capability tags, and strings, optionally including decompiled functions, and containing no category shortcut information.
On this textual state, the study set up three readout methods to evaluate each model:
- A0: The probability of a single judgment question, i.e., directly asking the model "is this malware?"
- A1: 32 atomic behavior questions, weighted and aggregated by a gradient boosting tree.
- A2: A classification head attached to the model's state embeddings.
The S1Mal approach feeds the atomic answers together with the A2 scores of CLM-8B into a gradient boosting tree. The evaluation covers CLM-8B, ten open decision models, and two LLMs read via logits, with datasets including the BODMAS temporal split, SOREL-20M out-of-distribution malware, and 14,534 matched benign files.
Ghidra is a disassembly and decompilation framework open-sourced by the U.S. National Security Agency, supporting multiple processor architectures and file formats. It can disassemble and decompile binaries to generate C-like pseudocode, and provides analysis views such as function call graphs and cross-references. capa is a malware capability identification tool open-sourced by Mandiant (now part of Google). It identifies high-level attack capability tags from binaries based on rule matching (e.g., "create process," "read file," "inject code"), and maps these capabilities to the MITRE ATT&CK framework. Combined, the two can transform raw binaries into structured, semantically readable textual states, providing input for LLMs.
MITRE ATT&CK is a public knowledge base of adversarial tactics and techniques, organizing specific attacker techniques by tactical stage (e.g., persistence, lateral movement, data exfiltration), with each technique having a unique identifier. In malware analysis, ATT&CK tags provide a standardized vocabulary of behavioral descriptions, enabling the atomic questions answered by the model to correspond to industry-standard threat language, making it easier for analysts to understand and relate to broader threat intelligence.
Key Findings: A0 Wastes Evidence, A1 and A2 Improve Substantially
On BODMAS, A0's performance exposes the limitations of a single judgment question: the AUROC of the 13 models spans from 0.4025 to 0.8590, indicating that static evidence is largely wasted.
A1 changed the picture. By decomposing the decision into 32 atomic behavior questions, each model's AUROC improved by an average of 54.61%. This result points to a core insight: having the model answer fine-grained behavior questions activates its internal knowledge more effectively than directly asking "is it malicious?"
A2 is similarly effective: CLM-8B's AUROC improved by 0.2749 to 0.9943, surpassing the summary-fact-based combiner (0.9901), and reaching a range of 0.9853 to 0.9948 across nine open models.
The S1Mal ensemble approach ultimately achieves an AUROC of 0.9946. On the tested ROC curve, at an actual FPR of 1.45% it detects 93.47% of malware; at the strict threshold of 0.1% FPR it detects 59.15%, compared to 46.59% with A2 alone.
Gradient Boosting Decision Tree (GBDT) is an ensemble learning method that serially trains a series of weak learners (usually shallow decision trees), with each new tree fitting the residuals of the preceding model, and the final prediction obtained by weighted summation. GBDT performs excellently on tabular features and small-to-medium-scale data, is insensitive to feature scaling, and can output feature importance, so it is often used as a meta-learner or ensemble aggregator. In this study, GBDT is used to aggregate the answers to the 32 atomic behavior questions (or the scores from the model embedding classification head) into a final judgment; its advantage lies in its ability to learn nonlinear combinations among different atomic questions while maintaining good interpretability.
Logits readout refers to directly taking the model's raw output scores on specific tokens (e.g., "yes" and "no") without generating text, and obtaining probabilities via softmax. Compared to generative readout, logits readout is more stable and computationally cheaper, but requires the model to have a clear probability distribution over the answer tokens, and these probabilities also need to be calibrated before they can be used for threshold decisions.
Implications for Security Operations
The value of this work goes beyond scores. Every S1Mal decision comes with the atomic answers that support it; analysts can see specifically which behaviors the model based its malicious conclusion on, rather than facing a black-box score. The probability outputs calibrated at a fixed FPR give threshold setting a clear risk meaning.
The first evaluation of System One decision models in malware detection shows that decomposing the judgment question into typed atomic questions and then aggregating them in an ensemble manner is an effective path to unlocking the detection capabilities of such models. For security scenarios that require both interpretability and calibratability, this direction deserves continued attention.
Related articles

CIO SmartKobe EX05 Review: A MagSafe Power Bank with Built-in Cable and Stand
CIO SmartKobe EX05 power bank review: built-in detachable Type-C cable, folding rotating stand, Qi 2.2 magnetic 25W wireless charging with 7.5W safe mode, 10000mAh dual semi-solid-state cells, pass-through charging and three-device simultaneous power—ideal for multi-device users who hate cables.

Nobel Laureates' Daring Self-Experiments: Betting Their Bodies to Rewrite Medical History
The official Nobel Prize video chronicles five laureates' self-experiments: de Hevesy's radioactive leftovers, Forssmann's self-catheterization, Tu Youyou's first dose, and Marshall's H. pylori gamble—how betting their bodies rewrote medical history.

Anthropic CEO Rings Nasdaq Bell: Reaffirms AI Won't Be Accelerated by Commercial Pressure
Anthropic CEO Dario Amodei reaffirmed at the Nasdaq bell-ringing that regardless of commercial or political pressure, the company will hold firm to setting the pace of AI development. This article examines the background, industry pressures, and potential impact.