#LLM inference performance
1509 related articles
·3 min
Apple Silicon LLM Inference Speed Benchmarks: A Complete Guide to Local Large Model Performance on Mac
In-depth analysis of Apple Silicon local LLM inference speed benchmarks covering M-series memory bandwidth, model quantization, MLX framework optimization, and Mac configuration guidance.
Read more →
·3 min
Local LLM Selection Guide: Practical Applications for Cybersecurity Purple Teams
A deep dive into how cybersecurity Purple Teams and SOC analysts can select locally deployed LLMs, covering hardware constraints, censored vs. uncensored models, specific recommendations, and RAG integration.
Read more →
·3 min
How Open-Source Models Achieve Superior Retrieval Performance at 1% of GPT's Cost
Deep analysis of how open-source models match GPT-level retrieval performance at 1/100th the cost. Covers RAG cost optimization, embedding model fine-tuning, and deployment strategies.
Read more →
·3 min
Distilling Linus's Code Review Philosophy from 32,000 Emails
The linus-torvalds-skill project distills Linus Torvalds's code review style from 32,000 kernel mailing list emails into an AI Agent-callable skill, with open pipeline and multi-model experiments.
Read more →
·3 min
Gemini 3.5 Real-Time Subtitles in Practice: Speech Transcription Performance in Esports Commentary
A developer tests Gemini 3.5 Live Translate's input transcription API for real-time esports subtitles, successfully recognizing game terms and player names in noisy League of Legends commentary.
Read more →
·3 min
Machine Learning Project Portfolio: A Complete Guide to Building Resume-Worthy ML Projects
From project selection to deployment, learn how to build resume-worthy ML projects. Covers end-to-end workflows, tiered project recommendations, and practical tips for ML learners transitioning from beginner to intermediate.
Read more →
·3 min
Qwen3 Series Deep Dive: The Multimodal Ambitions and Future Outlook of Open-Source LLMs
In-depth analysis of Alibaba's Qwen3 series, exploring its multimodal visual understanding, Chinese language capabilities, open-source ecosystem, and impact on developers and the AI industry.
Read more →
·2 min
Frame Selection Is All You Need: How to Make Large Models Truly Understand Video
Exploring the critical role of frame selection in video understanding systems, analyzing three strategies—uniform sampling, content-aware sampling, and query-driven selection—and their engineering implications.
Read more →
·2 min
Firstmate: Orchestrating AI Agent Team Collaboration Through a Single Entry Point
Deep dive into Firstmate's multi-agent collaborative development model: orchestrating a specialized AI team through a single conversational entry point, covering the full pipeline from requirements to delivery.
Read more →
·3 min
Feasibility Analysis: Deploying DeepSeek V4 Flash on Dual DGX Spark Systems
In-depth feasibility analysis of deploying DeepSeek V4 Flash on two NVIDIA DGX Spark units offline, examining memory bandwidth, MoE communication overhead, and quantization strategies.
Read more →
·2 min
Homebench: A Comprehensive Benchmarking Tool for Local LLMs Across Speed, Memory, and Quality
Homebench is an open-source local LLM benchmarking tool that evaluates models across speed, memory, and quality dimensions, helping developers make optimal model selection and quantization decisions.
Read more →
·2 min
Alibaba's Qwen Surges to #2 on Text Arena — A Deep Dive into the Model's Capabilities
Alibaba's Qwen LLM surges to #2 on Text Arena via blind human evaluation, showcasing top-tier alignment quality. Analysis of Qwen's technical strengths, open-source strategy, and industry impact.
Read more →
·3 min
Qwen3.8-Max Released: A New Benchmark in Autonomous Programming with 2.4 Trillion Parameters
Alibaba releases Qwen3.8-Max with 2.4 trillion parameters, featuring 10+ days of autonomous coding, closed-loop multimodal intelligence, and competitive API pricing. Open weights coming next week.
Read more →
·4 min
Guide to Building a Localized Document Intelligence System: From Architecture Design to Model Selection
A detailed guide on building a localized document intelligence system to replace Azure Document Intelligence for offline document parsing, covering layout analysis, OCR engine selection, multimodal LLM deployment, and hybrid solution design.
Read more →
·2 min
10 Trillion Tokens in 3 Months: Open-Source Model Demand Is Exploding
Open-source LLMs processed 10 trillion tokens in under 3 months, hitting 300B daily. We break down what this milestone means and why open-source demand is accelerating.
Read more →
·3 min
Building AI from Scratch: A Complete Learning Path with 503 Lessons in This Open-Source Course
AI Engineering from Scratch is an open-source course with 503 lessons across 20 phases, from linear algebra to autonomous agents, emphasizing hand-implementation before frameworks, supporting Python/TypeScript/Rust/Julia, with 46K+ GitHub stars.
Read more →
·5 min
Career Planning for ML Security Engineers: A Complete Learning Path from Scratch
A systematic career development guide for ML security engineers covering math foundations, ML core skills, and cybersecurity — with project ideas and learning resources for aspiring AI security professionals.
Read more →
·3 min
Unsloth Teams Up with Thinking Machines: Dynamic 1-bit Quantization Achieves 86% Compression, Running Multimodal LLMs in 270GB
Unsloth and Thinking Machines release dynamic 1-bit GGUF quantization for Inkling, compressing the model from 1.9TB to 270GB (86% reduction) while retaining 74.2% accuracy and adding vision/audio multimodal support.
Read more →
·2 min
hotcell: A Local Sandbox SDK Built for AI Agents
hotcell is an open-source self-hosted sandbox SDK designed for secure AI Agent code execution. Run locally on Mac/Linux with default-deny egress, per-sandbox ephemeral tokens, and resource controls.
Read more →
·4 min
Q8_K_XL vs. MXFP4 Naming Debate: The Quantization Truth That FP8 ≠ Q8_0
Deep dive into LLM quantization formats Q8_K_XL vs MXFP4, explaining why FP8 ≠ Q8_0, debunking the "8-bit is lossless" myth for local deployment users.
Read more →