[KongchangAI]
· 2 min read· 1,374 words

SAGA: Enhancing Molecular Language Model Representations with Classical Structural Priors

SAGA: Enhancing Molecular Language Model Representations with Classical Structural Priors

SAGA aligns molecular embeddings with fingerprint similarity via CKA, improving chemical fidelity.

This article introduces a study on representation learning for molecular language models. Researchers point out that models trained on structure-text pairs do not necessarily organize embeddings by chemical similarity. They propose SAGA, a training-only regularization method that aligns neural molecular representations with fingerprint-based Tanimoto similarity via Centered Kernel Alignment (CKA). SAGA adds no parameters and requires no fingerprints at inference. Experiments on MolCA and Atomas show improved molecule-text retrieval, better average performance on scaffold-split property prediction, and no degradation in molecule description or text-to-molecule generation. The method offers a general, low-cost way to inject classical cheminformatics priors, though fingerprint type effects and validation on larger models need further study.

Molecule-Language Models excel at multimodal tasks, but do the molecular representations they learn truly reflect structural similarity? A recent study points out that models trained on structure-text paired data do not necessarily organize their embedding space according to molecular fingerprint similarity. To address this, researchers propose SAGA (Structure-Aware Guided Alignment), a training-only regularization method that aligns neural molecular representations with fingerprint-based Tanimoto similarity via Centered Kernel Alignment (CKA), thereby improving the chemical fidelity of representations without adding parameters or requiring fingerprints at inference time.

rss source: Classical structural priors improve neural molecular representations in molecule language models

Why do molecular language models need structural priors?

Classical molecular similarity (e.g., fingerprint-based Tanimoto coefficient) is central to cheminformatics systems and widely used in virtual screening, clustering, and property prediction. However, molecular language models are typically trained via contrastive learning or masked language modeling, and their objectives do not explicitly constrain the embedding space to be consistent with chemical structural similarity. This leads to a potential issue: the model may map structurally dissimilar molecules to similar representations, or fail to distinguish subtle structural changes.

The authors note that even if a model performs well on molecule-text retrieval tasks, its embedding space may diverge from chemists' intuitive notion of "similarity." For example, two molecules with the same scaffold but different functional groups may not be closer in embedding space than two completely different molecules. This bias can limit the model's generalization on downstream tasks that require structural sensitivity (e.g., scaffold-split property prediction).

The motivation behind SAGA is to use a well-established cheminformatics prior—fingerprint similarity—as a supervisory signal to guide the model toward learning representations that better align with chemical intuition.

Molecular Fingerprint is a classical method that encodes molecular structure into a fixed-length vector. The basic idea is to systematically enumerate substructural fragments in a molecule (e.g., atoms, bonds, functional groups, ring systems) and map these fragments to vector dimensions via hashing or counting. Common fingerprint types include MACCS keys (166 bits) based on substructure keys, circular fingerprints based on the Morgan algorithm (e.g., ECFP4, typically 2048 bits), and topological path-based RDKit fingerprints. The Tanimoto coefficient measures the similarity between two fingerprint vectors, defined as the size of their intersection divided by the size of their union, ranging from 0 to 1, with values closer to 1 indicating greater structural similarity. Because fingerprints explicitly encode local structural features, Tanimoto similarity has long been regarded as one of the gold standards for measuring molecular structural similarity in cheminformatics, widely used in similarity searching for virtual screening, compound clustering, and similarity-based property prediction.

How SAGA works: CKA alignment and training regularization

The core of SAGA is Centered Kernel Alignment (CKA), a method for measuring similarity between two representation spaces. Specifically, during training, SAGA computes the discrepancy between CKA similarity among molecular embeddings and fingerprint Tanimoto similarity, and adds it as a regularization term to the loss function. Thus, while optimizing the original task (e.g., molecule-text contrastive learning), the model is also encouraged to maintain consistency between the embedding space and structural similarity.

The key design point is that SAGA introduces no additional parameters and does not require computing fingerprints at inference time. This means deployment still relies only on input molecules (e.g., SMILES or graph structures) without extra computational overhead. Regularization is applied only during training, making it easy to integrate into existing molecular language model training pipelines.

The paper validates SAGA on two models: MolCA and Atomas. MolCA is a multimodal molecule-language model, and Atomas is another type of molecular representation model. Experiments show that SAGA brings improvements on multiple tasks without compromising multimodal capabilities.

Centered Kernel Alignment (CKA) is a method for measuring similarity between two representation spaces, proposed by Kornblith et al. in 2019, originally used to analyze whether representations learned by different neural network layers or different models are similar. The core idea of CKA is: first center the two representation matrices (subtract the mean), then compute their kernel matrices (e.g., linear kernel or RBF kernel), and measure the alignment of the two kernel matrices via a normalized Hilbert-Schmidt independence criterion. CKA ranges from 0 to 1, with higher values indicating greater similarity between the two representation spaces. Compared to canonical correlation analysis (CCA), CKA is insensitive to rotation, scaling, and neuron ordering of the representation space, making it particularly suitable for comparing representations across models or modalities. In SAGA, CKA is used to measure the consistency between the neural molecular embedding space and the fingerprint Tanimoto similarity space, serving as a regularization signal to structure the embedding space.

Experimental results: retrieval, property prediction, and generation tasks

In molecule-text retrieval tasks, SAGA improves retrieval performance and pushes inexact retrieval results toward structurally closer analogs. This means that when the model cannot retrieve an exact match, the returned results are more chemically reasonable rather than random or irrelevant molecules.

In property prediction tasks, the paper adopts the scaffold-split evaluation protocol—where training and test sets are split by molecular scaffold—to test generalization to novel scaffolds. SAGA improves average property prediction performance, indicating that structural priors help the model learn more transferable representations.

Furthermore, in single-run ablation experiments, performance on molecule description generation and text-to-molecule generation tasks either improved or remained unchanged. This demonstrates that SAGA, as a regularization term, does not sacrifice the model's multimodal generation capabilities.

Scaffold-split is an important evaluation protocol in molecular machine learning. Its core idea is to split training and test sets by molecular scaffold, ensuring that scaffolds in the test set never appear in the training set. A molecular scaffold typically refers to the core ring system after removing side chains, extractable via the Bemis-Murcko algorithm. This splitting method simulates the generalization demand for novel chemical space in real drug discovery scenarios, as practical applications often require predicting new compounds that differ structurally from the training set. Compared to random splitting, scaffold-split better reflects the model's structural generalization ability but is also more challenging—the model cannot "cheat" by memorizing similar scaffolds. Therefore, performance improvements under the scaffold-split protocol typically indicate that the model has learned more transferable chemical representations rather than merely fitting the training data distribution.

Implications and limitations

The significance of SAGA lies in demonstrating how classical cheminformatics priors can be "injected" into modern molecular language models without changing model architecture or inference pipeline. This approach is general: any molecular language model trained on paired structure-text data can attempt to incorporate similar CKA regularization.

However, the paper leaves some open questions. For example, SAGA relies on fingerprint similarity as a prior, and different fingerprint types (e.g., MACCS, Morgan) may affect results, but the paper does not systematically compare them. Additionally, experiments are only conducted on MolCA and Atomas; effectiveness on larger-scale or different-architecture models remains to be verified. The statistical significance of the single-run ablations is also not fully discussed.

Overall, this work provides a concise and effective direction for representation learning in molecular language models: rather than relying entirely on data-driven approaches, it is better to introduce time-tested chemical structural priors so that the learned representations are more "chemically aware."

Note: This article is based on the paper abstract; please refer to the original for specific experimental details.

One-line Summary

SAGA aligns molecular embeddings with fingerprint similarity via CKA, improving chemical fidelity of representations.

Paragraph Summary

This article introduces a study on representation learning for molecular language models. The researchers point out that molecular language models trained on structure-text paired data do not necessarily organize their embedding space according to chemical structural similarity. To address this, they propose SAGA (Structure-Aware Guided Alignment), a training-only regularization method that aligns neural molecular representations with fingerprint-based Tanimoto similarity via Centered Kernel Alignment (CKA). SAGA adds no model parameters and requires no fingerprint computation at inference, making it easy to integrate into existing training pipelines. Experiments on MolCA and Atomas show that SAGA improves molecule-text retrieval performance, making inexact retrieval results closer to structural analogs; improves average performance on scaffold-split property prediction; and does not degrade performance on molecule description generation and text-to-molecule generation tasks. This method offers a general and low-cost approach to injecting classical cheminformatics priors into molecular language models, but the impact of fingerprint types and validation on larger-scale models require further study.

#molecular language models#SAGA#centered kernel alignment#molecular fingerprints#Tanimoto similarity#representation learning#cheminformatics
Share:

Related articles