#positional encoding
199 related articles
·3 min
NLP/ML PhD Job Search: Paid Resource Recommendations and Preparation Strategies for Research Scientist Interviews
A guide to paid resources for NLP/ML PhD students preparing for Research Scientist interviews, covering coding, ML fundamentals, system design, and mock interviews with budget allocation strategies.
Read more →
·4 min
Relearning Machine Learning: Lessons Learned and an Efficient Study Path from Those Who've Been There
If you could restart your ML journey, what would you do differently? This article covers the top 3 beginner mistakes, where to invest your time, and a proven efficient learning path.
Read more →
·3 min
Why Is Google the Only One Excelling at Video Understanding LLMs? The Triple Barrier of Data, Architecture, and Cost
Deep analysis of why Google Gemini leads in video understanding LLMs, covering YouTube data assets, native multimodal architecture advantages, and why OpenAI and Anthropic face compute cost and data barriers.
Read more →
·3 min
Must-Watch for Self-Learning AI: Top 10 YouTube Channels & A Complete Learning Path
A widely shared AI learning YouTube channel list from Reddit and X, covering 10+ quality channels from 3Blue1Brown to Andrej Karpathy, with a complete self-study learning path from math foundations to LLM engineering.
Read more →
·4 min
Transformer Attention Mechanism Explained: A Complete Breakdown from Word Vectors to QKV
A deep dive into how the Transformer attention mechanism works, covering word embeddings, embedding spaces, multi-head attention, and the Query-Key-Value mechanism with intuitive analogies.
Read more →
·3 min
Qwen3 Series Deep Dive: The Multimodal Ambitions and Future Outlook of Open-Source LLMs
In-depth analysis of Alibaba's Qwen3 series, exploring its multimodal visual understanding, Chinese language capabilities, open-source ecosystem, and impact on developers and the AI industry.
Read more →
·2 min
Why Are Users Demanding Kimi K3? A Reddit Rant Reveals the Truth About China's LLM Competition
A Reddit post sparks debate: users demand Kimi K3, citing DeepSeek's low prices. Deep analysis of Chinese LLM iteration speed, pricing strategies, and user loyalty.
Read more →
·2 min
Multilingual Relation Extraction in Practice: Label Leakage and Evidence Sparsity Pitfalls Explained
Deep analysis of two hidden pitfalls in multilingual relation extraction: label order leakage enabling model cheating, and evidence sparsity being more critical than label sparsity. Practical guide for GLiNER-style zero-shot model training.
Read more →
·3 min
Training a 1.3B Parameter LLM from Scratch: Complete Workflow and Core Challenges
A deep dive into the complete workflow of training a 1.3B parameter LLM from scratch, covering Transformer architecture design, data preparation, and distributed training optimization.
Read more →
·4 min
SELENE Open-Source Project: An Interactive Notebook Library for Systematically Learning AI and Machine Learning from Scratch
SELENE is an open-source AI learning resource built on Jupyter Notebooks, systematically covering ML, deep learning, Transformers, and LLMs with interactive code and math derivations for beginners.
Read more →
·4 min
17M-Parameter Transformer for Blood Glucose Prediction: Open-Source Diabetes AI Model Explained
An open-source blood glucose prediction model using BERT-style Transformer architecture with only 17M parameters, running on mobile devices with DILATE and Pinball loss for 2-hour glucose forecasting.
Read more →
·3 min
Reproducing GPT-2: How a Hidden Bug Affected Weight Training
An in-depth analysis of a hidden bug discovered while reproducing GPT-2 from scratch, revealing how implementation errors silently degrade weight quality and sharing practical debugging methodologies.
Read more →
·3 min
TokenTown: A Visual Approach to Understanding How LLMs Predict the Next Token
TokenTown is an open-source visualization project that intuitively presents the internal token prediction process of LLMs using a town metaphor. Learn its design philosophy and educational value.
Read more →
·3 min
Long Documents Don't Work: Why AI Agents Can't Follow Your Rules
Analysis of why AI Agents can't reliably follow long policy documents, covering context dilution, rule conflicts, and soft constraint limitations, with more reliable governance architectures.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Learn about its technical principles, performance gains, and value for long-context training and inference acceleration.
Read more →
·3 min
Moonshot AI Open-Sources FlashKDA: A Deep Dive into High-Performance CUDA Kernels for KDA Attention
Moonshot AI open-sources FlashKDA, providing high-performance CUDA kernels for Kimi Delta Attention. Explore its technical principles, performance gains, and value for long-context training and inference.
Read more →
·2 min
Transformer²: Achieving Co-Design of Robot Morphology and Control with a Unified Architecture
Deep dive into how Transformer² uses a unified Transformer architecture to integrate robot morphology design and motion control into one model, enabling task-driven end-to-end co-design for embodied AI.
Read more →
·3 min
Preparation Guide for NLP Research Scientist Internships: Key Focus Areas and Common Pitfalls
A comprehensive guide to preparing for NLP Research Scientist Intern roles, covering evaluation criteria, foundational knowledge, paper reading strategies, hands-on skills, and common pitfalls.
Read more →
·3 min
Kimi K3 Reaches 2.5T Parameters, MiniMax M3 Pro Brewing Simultaneously: China's LLM Arms Race Escalates
Kimi K3 adopts new architecture with 2.5T parameters and 1M token context. MiniMax M3 Pro targets 2.7-3T. OpenAI surpasses 7M users, HY-OCR 1.5 achieves SOTA, Amap launches world model.
Read more →
·3 min
Kimi K3 Hands-On Review: 2.5 Trillion Parameter Open-Source Model Challenges Claude
Kimi K3 hands-on review: Moonshot AI's 2.5T parameter MoE model matches Claude in coding, surpasses it in 3D game development, with API pricing at one-tenth the cost of competitors.
Read more →