Training a Transformer on Synthetic Data to Predict Blood Sugar: A Developer's T1DM Experiment

A developer shared on Reddit the second phase of their blood sugar prediction model. Unlike the first attempt, this time they relied entirely on synthetic data to train the model, then tested zero-shot performance on real CGM data — an approach with real practical value for addressing the data scarcity problem in medical AI.

From Real Datasets to Pure Synthetic Data

In the first phase, the author used three public Type 1 Diabetes (T1DM) datasets — OhioT1DM, ShanghaiT1DM, and AZT1D — to train an encoder-only Transformer model.

The key change this time: they built a T1DM patient simulator, used its output to train the model, then measured zero-shot performance directly on their own real Continuous Glucose Monitor (CGM) traces. In other words, the model had never seen any of the author's actual blood glucose readings before testing.

The value of this approach lies in the fact that high-quality labeled data is hard to obtain in medical settings and raises privacy concerns. If synthetic data generated by a physics/physiology-based simulator can transfer to the real world, it offers a viable path around the data bottleneck.

T1DM patient simulators are typically built on physiological/pharmacokinetic models, the most representative being the UVA/Padova model (also known as the FDA-accepted Type 1 Diabetes Metabolic Simulator, T1DMS). This model uses a set of differential equations to describe the glucose-insulin dynamic system, covering subcutaneous insulin absorption curves, hepatic glucose output, peripheral tissue uptake, and other physiological processes. Individual patient differences (such as insulin sensitivity and carbohydrate absorption rates) can be simulated by adjusting parameters. By sampling broadly across the parameter space, the simulator can generate a virtually unlimited number of "patient scenarios" and corresponding glucose trajectories, effectively compensating for the small scale and incomplete annotation of real clinical datasets. This type of simulator is also a common tool in closed-loop artificial pancreas (AP) system research — used to verify the safety of control algorithms on virtual patients before advancing to clinical trials.

Model Architecture: Lightweight by Design

This model is extremely small, with only 31,251 parameters, configured as follows:

• 16 layers
• 1 attention head per layer
• Hidden dimension of 16

Training was completed on an NVIDIA DGX Spark in under 60 minutes. This parameter count and training cost means the model is lightweight enough to deploy in resource-constrained environments like mobile devices.

Functionally, the model is an encoder-only Transformer that by default predicts blood sugar trends for the next 2 hours. Through an autoregressive approach, it can also perform long-horizon predictions — such as 8-hour overnight blood sugar forecasting — which is especially meaningful for risk scenarios like preventing nocturnal hypoglycemia.

Counterfactual Reasoning

The author specifically highlights that they trained the model's counterfactual reasoning capability. In the context of blood sugar management, this means the model can answer hypothetical questions like "if I inject a certain dose of insulin now, or consume a certain amount of carbohydrates, how will my blood sugar change?" This is an important step from pure trend prediction toward decision support.

Counterfactual reasoning has a rigorous definition in causal inference: given an observed outcome that has already occurred, asking "if some intervention variable had taken a different value, what would the outcome have been?" In the blood sugar prediction scenario, the model needs to jointly model the causal relationship between glucose dynamics and external interventions (insulin doses, carbohydrate intake, exercise intensity, etc.), rather than merely fitting statistical correlations in historical sequences. To give a neural network this capability, training data typically needs to explicitly construct "intervention-response" pairs: the simulator, starting from the same initial state, injects different intervention conditions and records the corresponding glucose trajectories. The model learns from this the direction and magnitude of intervention effects on the trajectory. This aligns with offline policy evaluation in reinforcement learning and the Potential Outcomes Framework in causal machine learning.

Real-World Deployment and Testing

Testing wasn't done in a lab environment — it ran on an Android app the author developed themselves, using ExecuTorch as the inference backend. ExecuTorch is PyTorch's on-device deployment runtime, confirming that this small model is genuinely targeting mobile deployment.

Testing covered data from three different CGM devices:

• Libre 3 Plus
• Anytime CT5
• Linx sensor

The data spans the past 30 days. Notably, the charts and data tables shown in the post all come from the base model without any LoRA adapters attached — purely zero-shot results.

LoRA Fine-Tuning

In practice, the author uses LoRA adapters within the app to lightly fine-tune the model, allowing it to gradually adapt to their real CGM traces. This combination of "synthetic data pre-training + on-device LoRA personalization" preserves generalizability while optimizing for individual differences — a pragmatic engineering solution.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning method. Its core idea: alongside the pre-trained weight matrix, attach a pair of low-rank matrices A and B (rank r is much smaller than the original dimension). During fine-tuning, only the parameters of these two small matrices are updated while the original weights remain frozen. At inference time, BA is added to the original weights, introducing no additional inference latency. In on-device scenarios, this means user devices only need to store and update a tiny number of "personalization parameters" (typically tens of KB), rather than retraining the entire model — dramatically reducing storage and compute overhead. For tasks with strong individual variation like blood sugar prediction, LoRA provides a privacy-friendly personalization path: user data stays on-device, adapters are trained locally, and nothing needs to be uploaded to the cloud.

Open Source Repositories and Significance

The author has fully open-sourced the entire project across three separate repositories:

• Model source code: github.com/0xdeadf1sh/T1DMAI
• Simulator source code: github.com/0xdeadf1sh/T1DMSIM
• Android app source code: github.com/0xdeadf1sh/T1DMDROID

The significance of this project lies not in parameter scale or SOTA performance, but in demonstrating a clear technical pipeline: use a physiological simulator to generate synthetic data for training a small Transformer, then apply on-device LoRA for personalized adaptation. For medical AI domains where data acquisition is difficult and privacy is sensitive, this "simulator-first" approach deserves more attention.

It's worth noting that this remains a personal experimental project. All results are based on data from a single user, and the model's generalizability and clinical reliability require larger-scale, more rigorous validation. It should not be used directly for real blood sugar management decisions.