Neural Network Embeds Every Font: Unexpectedly Generates a Flower-Shaped Structure

When Fonts Meet Neural Networks: A Chemical Reaction Between Vision and Algorithms

In the world of machine learning, "Embedding" is hardly a new term. From word vectors to image features, embedding techniques compress high-dimensional data into low-dimensional dense vectors, allowing computers to "understand" the similarity between objects. But when someone applied embedding techniques to every single font, the results were unexpectedly fascinating—a project on Reddit demonstrated that after embedding all fonts with neural networks, the resulting 2D structure was not only orderly but also accidentally formed the shape of a flower.

This project came from the Reddit community, and the author provided only a brief description: "Embedding Every Font with Neural Networks makes some Nice Structures (including a flower)." No lengthy paper, no complex formulas—but just the two keywords "every font" and "neural network embedding" were enough to spark the curiosity of tech enthusiasts.

What Does Font Embedding Actually Do?

To understand the brilliance of this project, you first need to grasp how fonts are "embedded."

The essence of a font is a collection of glyph outlines. In traditional methods, comparing two fonts often relies on manually designed features (such as stroke weight, slant, serif morphology). The neural network approach is: render each font as an image or extract contour point sequences, feed them into an encoder, and output a fixed-length vector. This vector is the font's "digital fingerprint."

The key point is that, if trained properly, distances in the vector space correspond to the visual similarity of fonts. That is, serif and sans-serif fonts will automatically separate, handwritten styles will cluster together, and blackletter and Song typefaces will maintain appropriate distances from each other. When these high-dimensional vectors are reduced to 2D or 3D for visualization, "Nice Structures" emerge—clusters, branches, continuous transitions, like a font evolutionary tree.

Why Does a "Flower" Shape Appear?

According to the project description, one of the embedded results has the shape of a flower. This was not deliberately designed but rather the product of the data distribution and dimensionality reduction algorithm working together.

One reasonable explanation is that the font space itself has a ring-like or radial topological relationship. For example, certain fonts form a closed loop along two dimensions—weight (thickness) and width—going from thin to thick and back from thick to thin, and different styles wrap around into a petal-like shape under 2D projection. Another possibility is that the author used nonlinear dimensionality reduction methods such as t-SNE or UMAP, which tend to preserve local neighborhood relationships. When there are multiple dense clusters in the data, a multi-petal structure resembling a flower emerges.

Some users in the Reddit comments also speculated that this flower might correspond to the extreme diversity of a certain category of fonts (such as monospaced fonts or handwriting fonts), which was "squeezed" into a symmetrical form in the embedding space. Whatever the specific reason, this shows that neural network embeddings can not only capture semantic similarity but also reflect hidden patterns in font design.

t-SNE and UMAP are two commonly used nonlinear dimensionality reduction methods. t-SNE (t-distributed Stochastic Neighbor Embedding) minimizes the KL divergence between conditional probability distributions of data points in high-dimensional and low-dimensional spaces. It is particularly good at preserving local neighborhood structures, so it tightly clusters similar data points together, but the global structure (such as distances between clusters) is often unreliable. UMAP (Uniform Manifold Approximation and Projection) is based on Riemannian geometry and topological data analysis. It better maintains the global structure while preserving local structure, and is usually faster to compute. Both are commonly used to project high-dimensional embedding vectors onto a 2D plane for visualization, but the same dataset may appear in different shapes with different methods—which is exactly why Reddit users were curious about which algorithm the author specifically used.

Technical Breakdown: From Fonts to Vectors to a Flower

Although the original post did not provide code details, based on common font embedding workflows, the general steps can be reconstructed:

Data Preparation: Collect as many font files (TTF/OTF) as possible, and render multiple characters (such as A-Z, 0-9) from each font as samples.
Feature Extraction: Use a Convolutional Neural Network (CNN) or Variational Autoencoder (VAE) to encode character images into vectors; some methods directly process contour point clouds.
Train the Embedding Space: Through contrastive learning or autoencoder reconstruction loss, make vectors of similar fonts close to each other.
Dimensionality Reduction Visualization: Use t-SNE, UMAP, or PCA to project high-dimensional vectors onto 2D, plot scatter points and color them.
Observe the Structure: In the scatter plot, fonts naturally cluster, and certain regions form special shapes—including that flower.

This process is not complex, but "embedding every font" means the data volume could be enormous. Google Fonts alone has over a thousand fonts, and including commercial fonts, the total can reach tens of thousands. Training an embedding model at such scale requires sufficient compute and storage. However, the author clearly managed it, and the results are visually delightful.

Similarities and Differences with Word Embeddings

Word embeddings (such as Word2Vec, GloVe) process discrete symbols, while font embeddings process continuous visual signals. Both share a core idea: map objects into a low-dimensional space so that similar objects are closer together. The difference is that the similarity in word embeddings is defined by context, while the similarity in font embeddings is defined by visual features. This also explains why font embeddings can produce more "geometric" structures—visual similarity naturally has continuity and symmetry.

Contrastive Learning is one of the core paradigms for training embedding spaces. Its basic idea is: construct positive sample pairs (similar fonts) and negative sample pairs (dissimilar fonts), and train the encoder so that positive pairs are close in vector space and negative pairs are far apart. Commonly used loss functions include Triplet Loss and InfoNCE loss. In font embedding tasks, positive sample pairs can be constructed from different character images of the same font, or the same character after slight data augmentation (such as rotation, scaling); negative sample pairs are different fonts. Through large-scale contrastive training, the encoder gradually learns to map visually similar fonts to nearby vector positions, thereby forming a meaningful embedding space structure.

Variational Autoencoder (VAE) plays an important role in font embedding. Unlike a regular autoencoder, VAE does not directly map the input to a fixed vector but instead maps it to a probability distribution (usually assumed to be Gaussian), from which a latent vector is sampled and then used to reconstruct the input. This probabilistic design brings two benefits: first, the latent space is more continuous and smooth, and adjacent vectors decode into font forms with smaller differences, making it suitable for interpolation; second, the latent space of VAE naturally has a regularization effect that can prevent overfitting. In font generation tasks, VAE can sample new vectors in the latent space and decode them into entirely new glyphs, which is precisely the technical basis for "font interpolation generation."

What Is This "Flower" Good For?

Some might ask: besides being visually appealing, what practical value is there in knowing what font embeddings look like?

First, font recommendation and retrieval. If you like a certain font in your design and want to find similar alternatives, the embedding space can directly compute nearest neighbors.

Second, font generation and interpolation. In the embedding space, vectors between two fonts can be interpolated to generate new fonts in intermediate forms. This provides new ideas for variable font design.

Third, font classification and anomaly detection. Automatically identify font categories (serif, sans-serif, handwriting, etc.), or discover fonts that are outliers in design.

Fourth, understanding the design space. The flower itself is a visualization of the topological structure of the font design space, helping designers and researchers understand the evolutionary relationships between fonts.

Many users on Reddit said that this kind of visualization gave them new intuition about font design. Some even suggested turning it into an interactive tool, allowing users to freely explore the font universe.

Community Response and Takeaways

This project received considerable attention on Reddit. Although the original post had limited information, the discussion centered on a few directions:

Choice of dimensionality reduction method: Some were curious whether t-SNE or UMAP was used, because different methods produce different shapes.
Number and diversity of fonts: The quality of the embedding results depends heavily on the dataset. If only a few fonts are included, the structure may not be obvious.
Reproducibility: Some users hoped the author would open-source the code so they could try it themselves.

From a broader perspective, this project once again proves the generality of neural network embeddings. Whether it's words, images, users, or fonts, as long as they can be vectorized, hidden structures can be discovered. And that flower is just a shell the algorithm偶然 scooped up from the ocean of data—it reminds us that machine learning can not only solve problems but also create aesthetic surprises.

If you're interested in font embedding, why not start by collecting a few dozen fonts, use an off-the-shelf CNN or VAE to extract features, and run t-SNE once. Perhaps you too can grow a flower of your own.