#256K context inference
81 related articles
·3 min
LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec
Learn why memory bandwidth (GB/s)—not VRAM size—determines local LLM inference speed. Get the tokens/sec formula, GPU bandwidth comparisons, and a practical card selection hierarchy.
Read more →
·3 min
LLM Inference Speed Formula: Memory Bandwidth Determines Your tokens/sec
Why memory bandwidth (GB/s), not VRAM size, determines local LLM inference speed. Includes tokens/sec formula, GPU bandwidth comparison, and a practical card selection framework.
Read more →
·3 min
Bo AI: Putting an AI Personal Assistant in Your Text Messages, Serving Everyday People with Zero Barriers
Bo AI is an AI personal assistant delivered through SMS, requiring no app download to manage schedules, save time, and get health advice. A deep analysis of its positioning and the SMS-first approach.
Read more →
·3 min
Bo AI: Putting an AI Personal Assistant in Your Text Messages, Serving Everyday People with Zero Barriers
Bo AI is an AI personal assistant that works through text messages—no app needed. It manages schedules, saves time, and answers questions. A deep analysis of its positioning, SMS-based approach, and the last-mile challenge of AI consumerization.
Read more →
·3 min
A Guide to Selecting Datasets for RAG Experiments: Evaluating Dense, Sparse, and Hybrid Retrieval
A systematic guide to standardized datasets for RAG retrieval experiments, covering BEIR, MS MARCO, Natural Questions, and TREC benchmarks for dense, sparse, and hybrid retrieval evaluation.
Read more →
·3 min
Anthropic Breaks HAWK-256: A Security Wake-Up Call for Post-Quantum Signature Schemes
Anthropic publishes a practical key-recovery attack on HAWK-256, exposing vulnerabilities in post-quantum signature schemes and implications for PQC standardization.
Read more →
·3 min
Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention
Deep dive into Kimi Delta Attention (KDA): from standard Softmax attention's quadratic bottleneck through linear attention, Delta Rule, and gated decay mechanisms — the complete evolution explained.
Read more →
·3 min
Kimi Delta Attention Explained: The Evolution from Standard Attention to Linear Attention
A deep dive into Kimi Delta Attention (KDA): tracing the evolution from quadratic Softmax attention through linear attention, Delta rules, and gated decay mechanisms, with insights on associative memory and hardware optimization.
Read more →
·3 min
Deep Dive into Inkling: The First Frontier Open-Source Multimodal LLM from the West
Thinking Machines releases Inkling, an open-source multimodal LLM with near-trillion MoE parameters, 1M token context, Apache 2.0 license. Deep dive into architecture, benchmarks, and pricing.
Read more →
·4 min
Colibri: Running a 744-Billion-Parameter LLM on an Ordinary Laptop
Colibri uses MoE hot-cold separation and 4-bit quantization to run 744B-parameter models like GLM 5.2 on consumer hardware. Learn about its three-tier memory architecture and speculative decoding.
Read more →
·3 min
Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide
Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.
Read more →
·3 min
Deep Dive into the Kimi-K3 Technical Report: Architecture, Long Context, and Reasoning Capabilities
Deep analysis of Moonshot AI's Kimi-K3 technical report covering long context processing, MoE architecture, reasoning capabilities, and China's position in the global AI competition.
Read more →
·3 min
Deep Dive into the Kimi-K3 Technical Report: A Comprehensive Analysis of Architecture, Long-Context Processing, and Reasoning Capabilities
Deep dive into Moonshot AI's Kimi-K3 technical report, analyzing its long-context processing, MoE architecture, reasoning improvements, and its position in global AI competition.
Read more →
·5 min
Ollama Performance Degrades After Connecting to OpenWebUI? A Complete Troubleshooting Guide
Complete guide to troubleshooting Ollama performance degradation with OpenWebUI, covering num_ctx truncation, sampling parameter differences, GPU resource contention, and system prompt interference.
Read more →
·3 min
Laguna S 2.1: The Open-Source Coding Model That's 14x Smaller Yet Stronger
Poolside releases Laguna S 2.1, an open-source agentic coding model: 118B total params with only 8B active, scoring 40.4% on DeepSWE — ~4.5x DeepSeek V4 Pro Max. Supports 1M-token context, deployable on a single workstation.
Read more →
·3 min
Multi-Agent AI Paper Writing System: A Full-Stack Breakdown of FastAPI + Vue3
A deep dive into an AI paper writing system built with FastAPI + Vue3, covering multi-agent collaboration, RAG, streaming output, and full academic workflow automation.
Read more →
·2 min
Agent A1 Open Source: 35B MoE Local Inference Speed Test & Deployment Guide
Agent A1 is a 35B MoE open-source model from Shanghai's Intern Science, activating only 3B params. Hits 95 tok/s on M4 Mac Studio. Supports 256K context, tool calling, and free local deployment via Ollama.
Read more →
·4 min
UK Sovereign AI: How Cosine AI Is Punching Above Its Weight Against American Compute Dominance
Cosine AI founder reveals how the UK's first sovereign LLM is being built — from government compute grants and RL credit attribution to multi-agent orchestration and synthetic data pipelines.
Read more →
·3 min
Qwen's Six-Generation Evolution: A Complete Breakdown from Architecture Innovations to the Reasoning Race
From Qwen1 to Qwen3-2507: a complete breakdown of six generations of evolution over two years — GQA, MoE, GRPO, dynamic thinking, and the core shift from architecture to reasoning.
Read more →
·3 min
Bernini Tile-Based Generation: An Engineered Approach to Video Super-Resolution in ComfyUI
Bernini is a ComfyUI video super-resolution node package using Tile Split/Select/Merge to solve seams, drift, and VRAM overflow. Benchmarked at 325s for 39 frames at 1920×1080.
Read more →