VX-One: How Typed Decision Heads Reshape Border Traffic Classification

When Signature Engines Meet Unknown Attacks: The Predicament of Border Classification

Border traffic classification has long faced an irreconcilable tension: signature engines are fast and format-safe, but almost completely blind to attacks they have never seen; generative models have strong semantic understanding, yet they reason slowly and produce outputs that are hard to parse, making them difficult to embed directly into real-time pipelines. Commercial products have already responded to this tension—according to the original article, by late 2026 a batch of commercial System One decision models (such as TypeSafe Jev and Microsoft-Decision-1) began selling typed decision primitives (binary gates, closed-set selection, ordinal scoring) as product APIs, yet academia lacks systematic research on such designs. VX-One attempts to fill this gap: a small multi-head network with 100K–10M parameters whose outputs are constrained to the three parseable primitives above, trained in a multi-task manner with per-head temperature calibration, achieving a p50 latency of roughly 0.7–1.7 ms on CPU.

Three-Head Output: Turning "Uncertainty" into a Programmable Interface

The core design of VX-One is not to pursue stronger representational capacity, but to structure the decision outputs.

Gating, Selection, and Scoring

The first head is a calibrated binary gate that answers "is this flow suspicious"; the second head makes a selection among closed-set candidates, and the candidate set always includes an explicit unknown option—this is crucial in security contexts, as it gives the model a legitimate outlet for "refusing to commit"; the third head outputs an ordinal score used for ranking and tiered handling. The three heads are trained jointly but each is temperature-calibrated independently, so that the confidence of each output can be separately interpreted and thresholded.

Why Typing Matters

The free-text outputs of generative models cannot be consumed directly in real-time pipelines, whereas typed primitives can naturally be parsed by downstream rule engines. VX-One's trade-off is to sacrifice some expressive freedom in exchange for deployment feasibility and auditability—which is also why commercial decision models choose the same route.

Temperature Scaling is a post-hoc calibration method: the model's output logits are divided by a learnable scalar temperature parameter T, then passed through softmax to obtain probabilities. When T>1 the distribution is smoother, and when T<1 it is sharper; T is fitted by minimizing the negative log-likelihood on the validation set. Its advantages are that it has very few parameters and does not change the model's ranking, only adjusting the absolute values of confidence. VX-One performs temperature calibration separately on the three heads, meaning that the gate's "suspicion probability," the selection's "candidate confidence," and the score's "grading reliability" can each be thresholded independently. However, temperature scaling assumes that the validation set and the deployment distribution come from the same source; once temporal drift occurs, the calibrated T becomes invalid—this is precisely the mechanism behind the ECE degradation from 0.04 to 0.40 in the article.

Experimental Results: Detection Wins, Attribution Loses

The paper compares against calibrated XGBoost, Random-Forest, MLP, and Kitsune on three benchmarks—UNSW-NB15, CIC-IDS2017, and EMBER2024—and measures a two-stage cascade of "Suricata (52,674 Emerging-Threats rules) → VX-One," running across five days on CIC-IDS2017.

The conclusions show a clear split: VX-One wins on the detection metrics (AUROC, FPR@95%TPR) on UNSW-NB15, and also leads under temporal drift on CIC-IDS2017 (0.711 vs. 0.637); but on the multi-class attribution task, XGBoost wins across all three benchmarks. This means the advantage of typed decision heads is concentrated on the "is it anomalous" judgment rather than fine-grained partitioning of "which type of attack it belongs to."

The Honesty of Unknown Attacks: 9/9 and a Gap Below Random

VX-One's claimed "unknown-attack honesty" performs extremely unevenly in leave-one-family-out tests. Across the 9 folds on UNSW, the gate successfully detects never-before-seen attack families in all cases, reaching 9/9; but on the CIC recon family, the detection rate falls below random level. That the same mechanism behaves so divergently across two benchmarks shows that the gate's generalization to "unknown" depends heavily on the data distribution itself.

Equally worth noting is the transferability of the abstention threshold: a threshold tuned on one benchmark performs poorly when moved to another. This directly undermines the "plug-and-play" deployment narrative—thresholds need to be recalibrated for the target environment.

Leave-One-Family-Out (LOFO) cross-validation is the standard protocol for evaluating generalization to unknown attacks: each time, one attack family (e.g., DoS, recon, exploits) is completely removed from the training set, the model is trained on the remaining families, and then tested on the removed family. Since the model has never seen any sample from that family during training, the test results reflect detection capability against "truly unknown" attacks rather than memorization of known classes. It is stricter than random splitting, because random splitting would let samples from the same attack family appear in both training and test sets, inflating the metrics. The number of LOFO folds equals the number of attack families; UNSW-NB15 has 9 attack families, hence 9 folds. The protocol's limitation is that if the removed family differs greatly in feature distribution from the retained families, the model may learn "out-of-distribution detection" rather than "attack detection," which explains why the same mechanism performs divergently across benchmarks.

The Gray Zone of the Cascade: Where Does 99.9% of Traffic Go?

In payload-free reconstruction scenarios, the signature engine can only parse about 0.1% of flows, with the gray zone covering roughly 99.9% of traffic. This figure vividly illustrates the coverage gap of pure signature solutions at modern borders. The cascade design lets the second stage take over the gray zone, achieving a combined recall of ≥0.978, validating the necessity of the two-stage architecture: the first stage handles rapid peak-shaving and format safety, while the second stage handles semantic fill-in.

Payload-free reconstruction refers to scenarios such as encrypted traffic, proprietary protocols, or incomplete fragment reassembly, where the signature engine cannot obtain application-layer payloads for pattern matching and can only rely on packet headers and flow statistical features (packet length sequences, inter-arrival times, direction ratios, etc.) to make judgments. Most of Suricata's 52,674 Emerging-Threats rules rely on feature strings in the payload, so in payload-free scenarios only about 0.1% of flows can be matched by rules, with the remaining 99.9% falling into the "gray zone"—neither judged malicious nor proven benign. The gray zone is not "safe traffic" but the blind spot of the signature engine, which is precisely why a second-stage semantic model is needed to take over. The core value of the cascade design is that the first stage handles the small amount of parseable traffic at extremely low cost while ensuring format safety, and the second stage uses a lightweight model to make binary judgments on the gray zone, thereby raising combined recall above 0.978 without significantly increasing latency.

Calibration Collapses Under Drift

The most severe finding concerns calibration. Under temporal drift, calibration collapses across all methods, with ECE degrading from 0.04 to 0.40, while ranking ability only partially survives. This undermines the premise that "calibrated confidence can be used directly for automated handling": the model's output probabilities are no longer trustworthy after distribution shifts, though relative rankings still have some reference value. For production systems, this means automated decisions based on absolute thresholds need to be paired with continuous monitoring and recalibration mechanisms.

Expected Calibration Error (ECE) measures the deviation between predicted probabilities and actual accuracy: predicted confidences are binned, and the weighted sum of the differences between average confidence and average accuracy within each bin is computed. ECE=0.04 means the average deviation is only 4 percentage points, so confidence is largely trustworthy; ECE=0.40 means that when the model says "90% suspicious," the actual hit rate may be only 50%, and absolute-threshold decisions would be severely inaccurate. ECE's drawbacks are that it is sensitive to the binning strategy and reflects only average deviation rather than direction, so it is often used together with MCE and Brier Score. The article's claim that "ranking ability only partially survives" means that metrics depending only on relative order, such as AUROC, decline less than ECE, because calibration drift mainly destroys the absolute values of probabilities, while ranking information is relatively more robust under distribution shifts.

Practical Implications

VX-One provides an honest measurement record: typed calibrated decisions have clear value in detection and gray-zone recovery, but are inferior to gradient-boosted trees in multi-class attribution, exhibit significant fragility in unknown-family generalization and threshold transfer, and have calibration that cannot be relied upon under drift. For engineering teams, a more pragmatic route may be—use signature engines as the first fast filter, use typed decision heads for gray-zone and binary judgments, delegate fine-grained attribution to tree models, and prepare drift monitoring and recalibration plans for all confidence outputs.

One-line Summary

VX-One replaces free-text outputs with typed decision heads, excelling at detection and gray-zone recovery, but exposing fragility in attribution, unknown generalization, and calibration.

Paragraph Summary

The article reviews VX-One, a small multi-head network with 100K–10M parameters: it constrains outputs to three parseable primitives—a calibrated binary gate, closed-set selection with an explicit unknown, and ordinal scoring—achieving a p50 latency of roughly 0.7–1.7 ms on CPU. Experiments show VX-One outperforms XGBoost on UNSW-NB15 detection metrics and under temporal drift on CIC-IDS2017, yet loses to XGBoost on multi-class attribution across all three benchmarks. Leave-one-family-out gating reaches 9/9 on UNSW but falls below random on the CIC recon family, and abstention thresholds transfer poorly across benchmarks. Cascade experiments show signature engines parse only about 0.1% of traffic in payload-free scenarios, while the two-stage combined recall reaches ≥0.978. Under temporal drift, calibration collapses across all methods, with ECE rising from 0.04 to 0.40.