#CBOW
3 related articles
·3 min
From Word Vectors to Large Language Models: The Complete Evolution of AI 2.0 Technology
From word vectors and embeddings to RNNs, BERT, Transformers, and ChatGPT — a complete guide to the technical evolution of large language models and the AI 2.0 era.
Read more →
·4 min
No MLP or Attention Required: How Point-Attractor Dynamics Learn Word Embeddings
A minimalist dynamical system experiment: without MLP, Transformer, or attention layers, point-attractor dynamics driven purely by co-occurrence pressure learns semantic similarity on SimLex-999.
Read more →
·7 min
Dispersion Loss: A Key Technique for Solving Embedding Condensation in Small Language Models
Embedding condensation is a hidden bottleneck in small language model training. Dispersion Loss combats this by enforcing representation spread during training at zero inference cost.
Read more →