#large model inference
3520 related articles
·3 min
SPA Tokenizer Fix Upgrade: Technical Analysis and Practice of a Wider Tokeniser
In-depth analysis of the SPA tokenizer fix and wider Tokeniser upgrade, exploring vocabulary expansion's impact on model performance, tokenizer mechanics, boundary handling fixes, and Playground verification.
Read more →
·4 min
Quantization Illustrated: A Deep Dive into Core Compression Techniques for Large Language Models
A deep dive into LLM quantization techniques covering symmetric/asymmetric quantization, PTQ, QAT, GPTQ, AWQ, and outlier solutions for efficient model deployment.
Read more →
·3 min
Deep Dive into vLLM Architecture: How PagedAttention Enables High-Throughput LLM Inference
Deep analysis of vLLM's high-throughput inference engine architecture, covering PagedAttention paging, KV Cache memory management, and continuous batching scheduling strategies.
Read more →
·3 min
picodl: Technical Analysis of a Deep Learning Library Built from Scratch with Pure NumPy
In-depth analysis of picodl, a lightweight deep learning library built from scratch with pure NumPy. Covers forward propagation, backpropagation, gradient computation, and discusses its educational value.
Read more →
·3 min
Why Is Google the Only One Excelling at Video Understanding LLMs? The Triple Barrier of Data, Architecture, and Cost
Deep analysis of why Google Gemini leads in video understanding LLMs, covering YouTube data assets, native multimodal architecture advantages, and why OpenAI and Anthropic face compute cost and data barriers.
Read more →
·2 min
AMD Acquires Taalas: What Does Etching AI Models Into Silicon Mean?
AMD acquires chip startup Taalas to etch AI models directly into silicon for extreme inference efficiency. We analyze the technology, tradeoffs, and AMD's differentiated AI strategy.
Read more →
·3 min
NVFP4 Dynamic Quantization in Practice: W4A4 Accelerated Deployment for the Full Gemma-4 Model Family
NVFP4 dynamic quantization covers all five Gemma-4 model sizes using W4A4 mixed-precision with calibrated FP8 KV Cache, dramatically reducing VRAM usage and deployment costs for efficient inference from edge to cloud.
Read more →
·4 min
Why CodeAct Code-First Agents Haven't Won Yet: A Deep Dive into the Paradigm's Dilemma
Deep analysis of why CodeAct code-first agents haven't replaced ReAct chat-first frameworks. Examining model training bias, protocol limitations, MCP design flaws, and sandbox challenges.
Read more →
·3 min
Qwen3-Max Deep Dive: How Coding and Collaboration Capabilities Are Redefining AI Development Assistants
Deep analysis of Alibaba's flagship model Qwen3-Max, covering its coding, Cowork collaboration capabilities, and potential for redefining AI-assisted software development.
Read more →
·3 min
Unsloth Releases Quantized DeepSeek V4 Flash: Lossless Performance at 162GB
Unsloth releases UD dynamic quantized versions of DeepSeek V4 Flash 0731, offering six variants from 162GB lossless to 83GB extreme compression using MXFP4+BF16 mixed precision.
Read more →
·3 min
Must-Watch for Self-Learning AI: Top 10 YouTube Channels & A Complete Learning Path
A widely shared AI learning YouTube channel list from Reddit and X, covering 10+ quality channels from 3Blue1Brown to Andrej Karpathy, with a complete self-study learning path from math foundations to LLM engineering.
Read more →
·3 min
Local LLM Selection Guide: Practical Applications for Cybersecurity Purple Teams
A deep dive into how cybersecurity Purple Teams and SOC analysts can select locally deployed LLMs, covering hardware constraints, censored vs. uncensored models, specific recommendations, and RAG integration.
Read more →
·3 min
How to Choose a Research Direction for Your AI Master's? A Practical Decision Framework for Employment
How should employment-focused AI master's students choose research directions? Analyzing action recognition, EEG image generation, affective computing, and causal inference from a skill transferability perspective.
Read more →
·3 min
Open-Source vs Closed-Source Models, Reward Hacking & Benchmark Gaming: A Deep Dive into Core AI Model Technical Issues
Deep analysis of six core AI model issues: open-source vs closed-source models, inference throughput vs accuracy tradeoffs, benchmark gaming, distillation vs RL, reward hacking defenses, and dynamic quantization technology.
Read more →
·2 min
Gemini 3.5 Pro Leak Rumors: Deep Analysis of Release Timing and Strategic Direction
Reddit claims Gemini 3.5 Pro is deployed and ready for release, but prediction markets and community remain skeptical. Deep analysis of leak credibility, Google's AI strategy, and the Pro vs Flash debate.
Read more →
·3 min
How Open-Source Models Achieve Superior Retrieval Performance at 1% of GPT's Cost
Deep analysis of how open-source models match GPT-level retrieval performance at 1/100th the cost. Covers RAG cost optimization, embedding model fine-tuning, and deployment strategies.
Read more →
·3 min
Distilling Linus's Code Review Philosophy from 32,000 Emails
The linus-torvalds-skill project distills Linus Torvalds's code review style from 32,000 kernel mailing list emails into an AI Agent-callable skill, with open pipeline and multi-model experiments.
Read more →
·2 min
Google AI Search Exposed for Gender Bias: Same Questions, Different Answers Based on Gender
Reddit users discovered Google AI gives different answers to identical questions based on gender — women's dating standards called 'personal preference' while men's are attributed to 'insecurity.'
Read more →
·2 min
Perplexity Enables 'Computer' Feature by Default, Silently Draining User Quotas and Triggering a Trust Crisis
Perplexity caught enabling Computer feature by default, silently draining Pro users' quotas. A deep dive into the trust crisis and AI monetization challenges.
Read more →
·2 min
When Claude Goes Down: The Anxiety Over Independent Thinking Behind AI Dependence
When AI services like Claude go down, dependent employees are lost while veteran colleagues think independently. Exploring the cognitive outsourcing risks behind AI dependence.
Read more →