#LLM inference ASIC
799 related articles
·3 min
The 5-Layer AI Tech Stack Explained: A Complete Industry Chain from Energy to Applications
Deep dive into the 5-layer AI tech stack: Energy, Chips, Infrastructure, Models, and Applications. Understand the key players, competitive landscape, and value distribution logic across the AI industry chain.
Read more →
·3 min
Quantprobe: An Open-Source Tool for Running 30B LLMs on 6GB GPUs
Quantprobe is an open-source memory optimization framework that enables 30B LLMs to run at 22 tokens/s on 6GB GPUs through per-layer quantization and intelligent CPU/GPU splitting.
Read more →
·2 min
Ollama Shifting from Local Deployment to Cloud Services? Community Controversy and the Open-Source Commercialization Dilemma
Ollama's recent brand shift from local LLM deployment to cloud API services sparks heated Reddit debate. Analyzing the capital logic, community concerns, and what open-source AI tool users should know.
Read more →
·4 min
Running an 80B Large Model on Mac with Only 4.3GB of Memory: Extreme Compression Techniques Explained
Deep dive into how an 80B-parameter LLM runs on Mac with only 4.3GB memory, covering ultra-low-bit quantization, sparsity, memory mapping, and implications for privacy and edge AI.
Read more →
·4 min
AI Engineer Growth Roadmap: From Programming Fundamentals to RAG and MCP Agent Development
A systematic AI engineer learning roadmap covering programming, math, ML, and data engineering foundations, plus frontier AI technologies like LLM, RAG, Agents, and MCP with free open-source resources.
Read more →
·3 min
Self-Learning NLP: A Complete Path from Zero to Practical Application
A complete self-learning path for NLP covering fundamentals, Transformer concepts, hands-on projects, and tools like Hugging Face to help developers master NLP without returning to school.
Read more →
·3 min
Nightcrawler: Deep Dive into an AI Penetration Testing Agent Running Locally on Smartphones
Deep analysis of Nightcrawler, an AI penetration testing agent running entirely on smartphones. Exploring how on-device AI empowers cybersecurity testing, its architecture, use cases, and risks.
Read more →
·4 min
From Zero to AI Engineer: A Real Growth Path from a Student's Resume
A CS student went from Python basics to model deployment in 3-4 months, building an AI portfolio through three real projects. This article breaks down the learning path, project value, and resume optimization strategies.
Read more →
·3 min
AI-Assisted Analysis Costs Drop 10x: The Tipping Point for Data Analytics Democratization
AI-assisted data analysis costs drop 10x: the technical logic and industry impact. From Text-to-SQL to compute cost declines, analyzing democratization trends, analyst role shifts, and deployment risks.
Read more →
·2 min
AI Agent Debugging Tool: Inspect Execution Chains Like Browser DevTools
Agent DevTools is an open-source AI Agent debugging tool inspired by Browser DevTools, offering execution visualization, tool call tracing, and breakpoint analysis to help developers diagnose Agent failures.
Read more →
·3 min
Control Theory Meets Machine Learning: A Learning Roadmap from Fundamentals to the Cutting Edge
A tailored ML guide for control theory learners covering reinforcement learning, data-driven control, Learning-based MPC, and a three-stage roadmap with practical advice.
Read more →
·4 min
SELENE Open-Source Project: An Interactive Notebook Library for Systematically Learning AI and Machine Learning from Scratch
SELENE is an open-source AI learning resource built on Jupyter Notebooks, systematically covering ML, deep learning, Transformers, and LLMs with interactive code and math derivations for beginners.
Read more →
·3 min
Building a Local AI Agent Architecture: From Scattered Scripts to Your Personal Jarvis
A practical guide to consolidating scattered automation scripts into a local AI Agent hub. Covers Function Calling, Ollama+Qwen2.5 deployment, tool orchestration architecture, and a complete implementation roadmap.
Read more →
·3 min
Fable-OS: A Self-Evolving Operating System That Writes Its Own Drivers
Fable-OS is an open-source self-evolving OS running on bare metal with natural language as its only interface. Its AI agent can autonomously write hardware drivers and evolve itself at runtime.
Read more →
·3 min
Dual RTX 3060 Running DeepSeek V4 Flash: IQ2_M Quantized Inference Benchmarked at 3.5 tok/s
Benchmarking DeepSeek V4 Flash on dual RTX 3060 GPUs with 96GB RAM at IQ2_M quantization achieving 3.5 tokens/sec. Covers hardware choices, 2-bit quantization techniques, and local LLM deployment optimization.
Read more →
·3 min
PyTorch vs TensorFlow: How Beginners Should Choose a Deep Learning Framework
Should deep learning beginners choose PyTorch or TensorFlow? This article compares both frameworks on research trends, ecosystem, and deployment, with practical switching advice.
Read more →
·5 min
Flux 3 Video Generation Experience: Home Movie Style and Prompt Optimization Capabilities
In-depth analysis of Flux 3 video generation model's home movie style capabilities, intelligent prompt optimization, Hermes Agent usage experience, and outlook for official release.
Read more →
·3 min
Ollama + Qwen-Agent: Building a Local Private AI Assistant — From Deployment to Security Best Practices
A complete guide to building a local private AI assistant with Ollama and Qwen-Agent. Covers RAG knowledge integration, voice interaction, and permission isolation for a secure local AI Agent architecture.
Read more →
·3 min
Reasoning Capabilities Unmatched, Pricing Still Too High: The Dilemma of AI Model Competition
Deep analysis of the dilemma in AI model competition where reasoning gaps and pricing imbalances force vendors to excel at either capability or cost-effectiveness to survive.
Read more →
·5 min
Have AI Doomers Actually Used AI? A Developer Exposes the Gap Between Hype and Reality
AI Doomers warn AI will destroy humanity, but have they actually built AI apps? A developer's sharp critique reveals the vast gap between AI demos and real engineering practice.
Read more →