Low-Data Molecular Generation: What You Randomize Matters More Than How Much
Randomizing SAFE alone boosts low-data molecular generation novel yield over 450-fold; dimension choice matters more than magnitude.
A controlled experiment studied representation augmentation for low-data molecular generation. Using 1000 seed molecules, researchers compared canonical vs. randomized SAFE and isolated atom-order and fragment-order randomization axes. Randomizing SAFE alone increased genuinely novel molecule yield over 450-fold, turning the model from a training-set repeater into a practical generator. The two axes show strong synergy—dual-axis randomization yields four times the sum of individual contributions—and randomized SAFE achieves 2.8x the true yield of randomized SMILES under matched corpora. The key insight: choosing which representation dimensions to randomize matters more than increasing randomization magnitude.
When training data consists of only a thousand molecules, chemical language models often just rehash the training set instead of learning a useful generative distribution. Randomized SMILES augmentation increases representation diversity by changing the atom traversal order, while SAFE (a fragment-based sequence representation) can additionally shuffle the order of fragment blocks. The impact of these two types of representation variation on low-data molecular generation had not been systematically studied before. A controlled experiment provides a clear answer: randomizing the SAFE representation alone can increase the yield of genuinely novel molecules by more than 450-fold, and which dimensions you randomize matters more than how much you randomize.
Experimental Design: Decomposing Randomization Dimensions with Controlled Variables
The researchers trained autoregressive sequence models on 1000 seed molecules, compared canonical SAFE with randomized SAFE at increasing corpus sizes, and further isolated two randomization axes: atom order and fragment order. For a fair comparison, they also constructed molecule-level matched training corpora to compare fully randomized SAFE with randomized SMILES. All main results were statistically validated using Welch's test and Games-Howell post-hoc comparisons to ensure the differences were not random fluctuations. This "molecule-by-molecule matching" design rules out interference from corpus composition differences, allowing the effect of the representation itself to stand out.
SMILES and randomized SMILES: SMILES (Simplified Molecular-Input Line-Entry System) is a text format that represents molecular structures as linear strings, generating sequences of atoms and bonds through depth-first traversal of the molecular graph. The same molecule can produce multiple valid SMILES due to different traversal starting points and branch orders—this is SMILES randomization: randomly selecting a traversal order during training to increase representation diversity for the same molecule. This augmentation strategy is simple and widely used, but it only changes the atom-level sequence order, not the way molecular fragments are combined.
SAFE representation and fragment order randomization: SAFE (Sequential Attachment Fragment Encoding) is a fragment-based molecular sequence representation that decomposes molecules into fragments (such as rings and functional groups) and records the connectivity between fragments. Unlike SMILES, SAFE sequences consist of fragment blocks, so in addition to randomizing the traversal order of atoms within fragments, the order of fragment blocks in the sequence can also be randomly shuffled. This dual randomization axis—atom order and fragment order—provides the model with richer structural variation signals and is the core representation-level augmentation approach examined in this study.
Finding 1: Randomizing SAFE Alone Yields Over 450x Improvement
The most direct conclusion is that randomizing the SAFE representation transforms the model from a "parrot" into a practical molecular generator. Under baseline conditions, the model tends to exactly reproduce training molecules; after introducing randomization, the yield of genuinely novel (non-training-set) molecules increased by more than 450-fold. This means that representation-level augmentation strategies alone are sufficient to change the model's behavior patterns without increasing data volume or model capacity. For drug discovery scenarios with only a small number of known molecules, this improvement has practical significance—it makes generative design feasible under low-data conditions.
Finding 2: The Two Randomization Axes Act Synergistically, Not Simply Additively
Randomization is not a linear game of "the more, the better." The study decomposed SAFE's two randomization axes: atom order and fragment order. Randomizing atom order alone achieved only one-tenth of the true yield obtained when both axes were randomized simultaneously; randomizing fragment order alone achieved only one-seventh. The yield from varying both axes simultaneously was four times the sum of their individual contributions. This indicates a strong synergistic effect between the two axes: shuffling fragment order may alter the fragment combination patterns the model learns, while varying atom order increases the diversity of local representations—only together can they unlock the maximum generative potential. Simply increasing randomization intensity is less effective than choosing the right randomization dimensions.
Synergy: In statistics and experimental design, synergy refers to the effect produced when two or more factors act together being greater than the sum of their individual effects. This study found that when atom order randomization and fragment order randomization are enabled simultaneously, the yield of genuinely novel molecules is four times the sum of their individual contributions, indicating that the two are not independently additive but mutually enhancing. A possible mechanism: shuffling fragment order forces the model to learn the combinatorial rules between fragments, while varying atom order increases the diversity of intra-fragment representations; together they enable the model to cover a broader generation space, thereby more effectively avoiding simple repetition of the training set. This finding suggests that in representation augmentation strategies, interactions between dimensions may be more critical than the intensity of any single dimension.
Finding 3: With Matched Corpora, Randomized SAFE Outperforms Randomized SMILES
Under conditions of matched corpus size and computational cost, the true yield of randomized SAFE was 2.8 times that of randomized SMILES. SMILES randomization only changes atom traversal order, while SAFE simultaneously shuffles fragment block order, providing richer structural variation signals. This comparison directly answers the question of whether representation format matters: in the low-data regime, which representation you choose to apply randomization to determines the quality of generation results more than the magnitude of randomization. For practitioners, this means that when data is scarce, prioritizing fragment-based representations (such as SAFE) and enabling their full randomization capabilities may be more effective than simply increasing the diversity of SMILES augmentation.
Practical Implications: Low-Data Generation Should Prioritize "Randomizing the Right Dimensions"
The core message of this research is: in low-data molecular generation, the design of augmentation strategies needs to shift from "how much to randomize" to "what to randomize." The synergistic effect of atom order and fragment order shows that structural changes at the representation level are more valuable than simple sequence perturbations. For researchers and engineers using chemical language models, the recommendations when data is limited are: first, prioritize representations that support fragment-level randomization (such as SAFE); second, enable both atom order and fragment order randomization simultaneously rather than choosing only one; third, when evaluating augmentation effectiveness, focus on the yield of genuinely novel molecules rather than just loss or perplexity metrics. These findings provide a low-cost, high-reward optimization direction for molecular generation under small-data conditions.
One-line Summary
Randomizing the SAFE representation alone can boost the novel yield of low-data molecular generation by over 450-fold, and the choice of randomization dimensions matters more than the magnitude.
Paragraph Summary
A controlled experiment systematically studied representation augmentation strategies for molecular generation models under low-data conditions. Researchers trained autoregressive sequence models on 1000 seed molecules, compared canonical SAFE with randomized SAFE, and isolated two randomization axes: atom order and fragment order. Results show that randomizing SAFE alone increases the yield of genuinely novel molecules by more than 450-fold, transforming the model from a training-set repeater into a practical generator; the two randomization axes exhibit strong synergy, with simultaneous dual-axis randomization yielding four times the sum of their individual contributions; and under matched corpora and computational cost, randomized SAFE achieves 2.8 times the true yield of randomized SMILES. The core insight: in low-data generation, choosing which representation dimensions to randomize is more important than simply increasing randomization magnitude, and practitioners should prioritize representations supporting fragment-level randomization while enabling both axes simultaneously.
Related articles
world2vec: Building Mechanistic World Models with Latent Dynamics Learning
world2vec proposes a reward-free method for learning mechanistic world models via latent dynamics, faithfully reproducing systems over long horizons with at least an order of magnitude fewer parameters than DreamerV3, TD-MPC2, and LeWM, and achieving accurate system identification on 71 attractor benchmarks.
A New KL-DRO Paradigm: Adaptive Scenario Generation Around a Learned Density Model
KL-DRO research proposes centering the KL ball on a learned density model instead of the empirical distribution, with adaptive scenario generation guiding sampling toward the worst case. Experiments show lower cost under distribution shift when records are scarce and the model can extrapolate, with comparable in-distribution cost.
ODE Samplers for Discrete Diffusion Models: Exponential Residual Flows Accelerate Generation
Discrete diffusion models lack deterministic ODE sampling? This article introduces Exponential Residual Flows (ERFs), which introduce continuous residual variables per token to construct piecewise ODEs for deterministic sampling and support consistency distillation acceleration. Superior to prior methods on code and text generation tasks.