[KongchangAI]
· 1 min read· 961 words

Personal Blood Glucose Prediction Model: A 30K-Parameter Transformer with LoRA Fine-Tuning in Practice

Personal Blood Glucose Prediction Model: A 30K-Parameter Transformer with LoRA Fine-Tuning in Practice

A 30K-parameter Transformer with synthetic pretraining and LoRA fine-tuning achieves personal glucose prediction rivaling published models.

A developer trained a 31,253-parameter encoder-only Transformer on ~19,200 CGM-years of synthetic data, then LoRA-fine-tuned on 115 days of his own CGM data. Inputs include insulin and carbohydrate action curves for counterfactual reasoning; outputs are 2-hour glucose forecasts in Kovatchev risk space with a DILATE+MSE+pinball loss. The model imputes missing data, backtests, and learns time-of-day with ≥48h context. It still trails the developer at 30-minute hypoglycemia prediction, and counterfactual reasoning lacks benchmarks. Code is open source.

A developer used a 30K-parameter Transformer model to predict his own blood glucose, achieving results comparable to published models. This personal experiment demonstrates the potential of synthetic-data pretraining combined with LoRA fine-tuning for medical time-series forecasting.

Reddit original post screenshot

Model Architecture and Training Strategy

The developer trained an encoder-only Transformer with just 31,253 parameters. The core idea is "synthetic-data pretraining + personal-data fine-tuning." The model was first pretrained on roughly 19,200 CGM-years (continuous glucose monitoring years) of synthetic data—records from 1 million simulated type 1 diabetes patients, 7 days each. He then used his own 115 days of CGM data (80 days for training the adapter, 35 days for testing) for LoRA fine-tuning.

The advantage of this two-stage strategy: synthetic data lets the model learn general glucose dynamics, while LoRA fine-tuning adapts it to individual characteristics without overfitting. According to the developer, the model is competitive with other published models.

LoRA (Low-Rank Adaptation) is a parameter-efficient fine-tuning technique: it freezes all original weights of the pretrained model and injects trainable low-rank matrix pairs (typically a down-projection matrix A and an up-projection matrix B) into the Transformer's attention layers, updating only these newly added parameters. Since the rank r is much smaller than the original dimensions, the number of trainable parameters often drops to 0.1%–1% of the original model, with memory and storage costs falling dramatically as well. For scenarios like personal health data—small sample sizes and privacy-sensitive—LoRA preserves the general glucose dynamics learned from synthetic data while adapting to individual characteristics with very few parameters, significantly reducing overfitting risk. After fine-tuning, the adapter can be saved and swapped independently, making it easy to reuse the same base model across different individuals or devices.

Multi-Input and Output Design

The model accepts not only historical CGM time series but also two additional inputs: insulin (U) and carbohydrates (G, GI), provided as action curves. Insulin dose U is an exact value, while carbohydrate intake (G, GI) is the developer's best estimate. This design enables counterfactual reasoning—for example, "what would happen to blood glucose if I took 1 more unit of insulin?"

On the output side, the model predicts blood glucose for the next 2 hours, represented in Kovatchev risk space. The training loss combines DILATE, MSE, and pinball loss, fused via learned Kendall-Gal weights. In addition, the model can impute missing data and perform backtesting (predicting the past).

The developer notes that the model also learned to predict time of day from historical glucose—a capability that emerged with context windows ≥48 hours. The model natively supports variable context windows from 3.5 to 7.0 days.

Kovatchev risk space is a nonlinear transformation framework for assessing glycemic control: it maps raw glucose values to risk values, assigning a steeper penalty slope to the hypoglycemic range than to the hyperglycemic range, thereby reflecting the acute danger of low blood glucose. Common forms include the Glucose Risk Index (GRI) and low/high blood glucose indices. Making predictions and computing losses in this space makes the model more sensitive to dangerous hypoglycemic excursions and better aligned with clinical concerns.

DILATE loss is a loss function for time-series forecasting that combines two differentiable components—shape and temporal distortion—allowing predictions to shift moderately along the time axis without being over-penalized. It suits signals like glucose curves that are sensitive to phase errors but whose morphology matters; pinball loss (quantile loss) is used to train quantile predictions, providing an interval rather than a point estimate for future glucose.

Limitations and Personal Experience

Despite the strong performance, the developer admits the model is still worse than himself at predicting hypoglycemia about 30 minutes ahead. This may be because hypoglycemic events involve complex individual physiological factors that are hard to fully learn from data.

Currently, there is no benchmark to validate the counterfactual reasoning capability, but his intuitive experience using the model on his phone is that when future insulin and carbohydrates are forcibly taken into account, the model responds sensibly. The source code is open-sourced on GitHub (0xdeadf1sh/T1DMAI) for interested developers.

This personal experiment shows that a small-scale Transformer combined with LoRA fine-tuning can reach a practical level on specific health prediction tasks, offering a low-cost, reproducible path for personalized medical AI.

Counterfactual reasoning here refers to the model's ability to predict how the glucose trajectory would change given different interventions (e.g., an extra 1 unit of insulin or a certain carbohydrate intake). Unlike ordinary prediction, it requires the model to learn a causal-style response between actions and outcomes, rather than merely fitting historical correlations. Since multiple interventions at the same moment cannot be observed simultaneously in the real world, counterfactual predictions lack direct labels and can usually only be validated indirectly via simulators or intervention trials—which is why the developer currently has no benchmark.

One-line Summary

A developer used a 30K-parameter Transformer with synthetic pretraining and LoRA fine-tuning to achieve personal glucose prediction comparable to published models.

Paragraph Summary

A developer trained an encoder-only Transformer with just 31,253 parameters to predict his own blood glucose. The model was first pretrained on roughly 19,200 CGM-years of synthetic data to learn general glucose dynamics, then adapted to the individual via LoRA fine-tuning on his own 115 days of CGM data. Inputs include insulin and carbohydrate action curves in addition to historical CGM, enabling counterfactual reasoning; outputs are glucose for the next 2 hours in Kovatchev risk space, with a loss combining DILATE, MSE, and pinball. The model can also impute missing data, backtest, and—with context ≥48 hours—exhibit an emergent ability to predict time of day. Limitations: it is still worse than the developer himself at predicting hypoglycemia about 30 minutes ahead, and counterfactual reasoning lacks benchmark validation. The code is open-sourced, demonstrating a low-cost, reproducible path for personalized medical AI.

#blood glucose prediction#Transformer#LoRA fine-tuning#continuous glucose monitoring#synthetic data pretraining#personalized healthcare AI#counterfactual reasoning
Share:

Related articles