#TensorRT inference
124 related articles
·3 min
ComfyUI-INT4-Fast: Running Flux Models Smoothly on 6GB GPUs
ComfyUI-INT4-Fast brings W4A4 quantized inference to ComfyUI. RTX 3060 (6GB VRAM) generates 1024×1024 images in 17s. Per-layer mixed-precision routing balances speed and quality for Flux models.
Read more →
·5 min
Ollama Raises $65M Series B: 85% of Fortune 500 Have Already Deployed Local LLMs
Local LLM tool Ollama closes a $65M Series B, bringing total funding to $88M. With 9M developers and 85% of Fortune 500 having deployed internally, this deep dive explores why enterprises embrace local LLMs: compliance, Agent cost savings, and open-source ecosystem.
Read more →
·3 min
From IT Support to AI Engineer: A Real NCA-GENL Certification Journey and Lessons Learned
A real NCA-GENL study journal from an IT-support-turned-AI-engineer: 50+ scenario questions, 7-week prep, and a brutal 40% on Trustworthy AI. Covers Transformer concepts, NVIDIA tools, and what actually works.
Read more →
·7 min
50 AI Research Job Applications to FAANG, 0 Replies? Unpacking the Truth Behind the Job Search Struggle
An AI research engineer with 3 years of experience sent 50 applications to FAANG with zero replies. This article breaks down the hidden barriers of top-tech AI roles, the truth about LinkedIn ghost jobs, and the MLE vs. Research Engineer divide.
Read more →
·5 min
GPT-5.6 Trio Released: A Deep Dive into Sol/Terra/Luna and the Breakthrough in Design Judgment
OpenAI officially releases the GPT-5.6 series with three models: flagship Sol, balanced Terra, and economy Luna. A deep dive into its core breakthroughs—a step change in design judgment and enhanced computer-use capabilities.
Read more →
·4 min
Kun: An Open-Source AI Coding Agent Deeply Optimized for DeepSeek
Kun is an open-source AI coding agent optimized for DeepSeek and domestic users, with nearly 5,000 GitHub stars. Features include requirements drafting, inline diffs, cost visualization, and mobile monitoring. Real-world cache hit rates reached 97%, keeping costs extremely low.
Read more →
·6 min
Tencent Hunyuan HY3 Official Version Open-Sourced: The Strongest Under 300B, at 1 Yuan per Million Tokens
Tencent Hunyuan HY3 official version is open-sourced under Apache 2.0, priced as low as 1 yuan per million input tokens, with major gains in agents, reasoning, coding, and long context. On the same day, Meituan open-sourced its trillion-parameter LongCat 2.0.
Read more →
·5 min
DeepSeek Boosts Inference Speed by 85%: How Speculative Decoding Is Reshaping the AI Speed Race
DeepSeek and Peking University's DS Spark paper boosts AI inference speed by up to 85% via confidence scheduling and semi-autoregressive speculative decoding — no model or GPU changes.
Read more →
·7 min
Krea2 Tested: Does the FP8 vs BF16 Quality Gap Still Exist?
A Reddit user compared FP8 and BF16 precision on the Krea2 model and found almost no difference in image quality. This article analyzes the technical reasons behind the shrinking quantization gap and its real-world impact on VRAM usage and inference speed.
Read more →
·7 min
The Agent Engineer Transition Roadmap: A Deep Dive into Three Core Skill Systems
Want to become an Agent engineer? This article systematically covers three core skill tracks—LLM fundamentals, LangChain architecture development, and enterprise deployment—to help you avoid detours.
Read more →
·6 min
Xiaomi XiaoAI 10.1-inch Smart Control Panel: Is the AI LLM Integration Worth It?
Xiaomi XiaoAI 10.1-inch Smart Control Panel features AI LLM Q&A, WeChat calling, and whole-home Mi IoT control. Priced at 839 yuan, ~679 yuan after national subsidy. An in-depth review of AI capabilities, screen experience, and smart home integration.
Read more →
·6 min
AMD Ryzen AI Halo Dev Kit: A $4,000 Local LLM Powerhouse Taking on Apple and NVIDIA
AMD officially unveils the Ryzen AI Halo local AI dev kit, priced around $4,000 with 128GB unified memory, capable of running 70B LLMs locally. An in-depth look at its specs, pricing, and market competition.
Read more →
·5 min
DeepSeek DSpec Hands-On: How Speculative Decoding Accelerates LLM Inference by 6x
Hands-on report on DeepSeek's open-source inference acceleration toolkit DSpec: draft model + smart scheduling delivers lossless speedup, hitting acceptance length 6 on GSM8K and reproducing official data.
Read more →
·4 min
YOLO Learning Roadmap: From Version Evolution to Source Code Deep Dive
A systematic YOLO learning roadmap: from understanding V1/V3/V4 version evolution, to building knowledge via video, to mastering implementation by debugging source code.
Read more →
Google Drops Two New Models: 4-Second …
·4 min
Google Drops Two New Models: 4-Second Image Generation + Real-Time Video Creation Explained
Google launches Imagen 3 Nano (Flash) for 4-second text-to-image generation and Veo 3 Flash for conversational video editing — now available via Gemini API and Google AI Studio.
Read more →
·4 min
NVFP4 Quantization in Practice: A Guide to Compressing Large Models with NVIDIA Model Optimizer
Deep dive into NVFP4 quantization: using NVIDIA Model Optimizer to compress Nemotron 3 Ultra to FP4 checkpoints, reducing memory by 75% and boosting inference throughput on Blackwell GPUs.
Read more →
·5 min
NVIDIA AI-Q Blueprint × OCI: A Practical Guide to Production Deployment of Enterprise AI Agents
Deep dive into NVIDIA AI-Q Blueprint production deployment on Oracle Cloud Infrastructure, covering NIM microservices, RAG architecture, multi-agent orchestration, and OCI GPU selection for enterprise AI agents.
Read more →
·5 min
NVIDIA AI-Q Blueprint × OCI: A Practical Guide to Production Deployment of Enterprise AI Agents
Deep dive into NVIDIA AI-Q Blueprint production deployment on Oracle Cloud Infrastructure, covering NIM microservices, RAG architecture, multi-agent orchestration, and OCI GPU selection.
Read more →
·3 min
AMD Ryzen AI Halo Local AI Dev Kit Deep Dive: Is $4,000 Worth It?
AMD Ryzen AI Halo dev kit at $4,000 features 128GB unified memory and XDNA 2 NPU for local LLM inference. Deep dive into architecture, performance trade-offs, vs. Mac Studio, and software ecosystem challenges.
Read more →
·6 min
14x Speedup: How Manticore Rebuilt the ONNX Embedding Inference Path
Manticore Search restructured its ONNX inference path to achieve 14x faster text embeddings. Deep dive into batching, session reuse, zero-copy memory, and thread tuning for vector search systems.
Read more →