#FP8 FP4 computing
227 related articles
·2 min
Chrome Built-in AI Explained: How Locally Running Models Protect Your Privacy
Explore Chrome Built-in AI technology and how running AI models locally in the browser enables zero data upload, instant responses, and stronger privacy protection.
Read more →
·2 min
Chrome Built-in AI Explained: How Locally Running Models Protect Your Privacy
Deep dive into Chrome Built-in AI technology, exploring how running AI models locally in the browser achieves zero data uploads, instant responses, and stronger privacy protection.
Read more →
·3 min
Monocular RGB Human Mesh Reconstruction: Real-Time Motion Capture at 18 FPS with Rust + CUDA
A deep dive into a real-time 3D human mesh reconstruction project using a single RGB camera, built with Rust, Candle, and CUDA, achieving 55ms/frame on an RTX 5080.
Read more →
·3 min
Monocular RGB Human Mesh Reconstruction: Real-Time Motion Capture at 18 FPS with Rust + CUDA
Deep dive into a real-time 3D human mesh reconstruction project using a single RGB camera, built with Rust, Candle, and CUDA, achieving 55ms/frame on RTX 5080. Exploring its architecture, Metal porting plans, and applications in VTuber, AR/VR, and sports analysis.
Read more →
·3 min
AMD CDNA5 Architecture Deep Dive: Technical Evolution and the AI Computing Competition Landscape
Deep analysis of AMD's CDNA5 architecture covering Chiplet packaging upgrades, HBM memory evolution, and low-precision compute optimization, examining how AMD challenges NVIDIA's AI chip dominance.
Read more →
·3 min
Kimi K3 Launches on Ollama Cloud: Free Tier, User Experience, and Developer Guide
Kimi K3 officially launches on Ollama Cloud as an "extra high usage" model. This guide covers free tier quotas, cloud inference experience, technical advantages, and how developers can seamlessly call this high-performance LLM.
Read more →
·3 min
DeepSeek-OCR Deployment & Fine-Tuning in Practice: From vLLM Inference to Unsloth Fine-Tuning
Complete guide to DeepSeek-OCR from vLLM inference deployment and Unsloth model loading to fine-tuning, covering cloud server setup, GPU selection, and code examples — all on a single 4090 GPU.
Read more →
·2 min
RX 9060 XT vs RTX 5060 Ti: How to Choose a GPU for Local AI
RX 9060 XT vs RTX 5060 Ti both offer 16GB VRAM — which is better for local AI inference? A full comparison of CUDA ecosystem, ROCm compatibility, LLM performance, and real-world usability.
Read more →
·2 min
RX 9060 XT vs RTX 5060 Ti: Which GPU to Pick for Local AI?
RX 9060 XT vs RTX 5060 Ti — both 16GB VRAM, but which is better for local AI? We compare CUDA ecosystem, ROCm compatibility, LLM inference, and real-world usability.
Read more →
·1 min
Radxa Cubie A7Z Real-World Test: 330ms End-to-End Object Detection Latency Under Full NPU Load
Deep analysis of Radxa Cubie A7Z achieving 330ms glass-to-glass object detection latency under full NPU load, covering latency breakdown, practical use cases, and edge AI optimization strategies.
Read more →
·3 min
Building a YOLO26 Inference Engine from Scratch in ARM64 Assembly: A Deep Dive into Low-Level Optimization
Deep dive into building a YOLO26n object detection inference engine from scratch using ARM64 assembly and C, covering NEON SIMD, Winograd convolution, GEMM micro-kernels, and cache tiling optimizations.
Read more →
·4 min
Remote Surgery at 199ms Latency? What Network Latency Really Means Across Different Scenarios
Gamers struggle above 50ms, yet remote surgery works safely at 199ms latency. This article explains why, covering jitter stability, motion prediction algorithms, and dedicated medical networks.
Read more →
·3 min
Kimi K3 Local Deployment Guide: Storage Requirements for q4 and q8 Quantization Explained
Detailed analysis of Kimi K3 quantization deployment options, comparing q4 vs q8 storage requirements, precision trade-offs, and hardware configurations for local self-hosting.
Read more →
·7 min
DIY Companion Robot: Building Your AI Desktop Buddy with Jetson Orin
A maker builds a DIY companion robot with NVIDIA Jetson Orin and 4S LiPo battery. Explore the full development journey from first power-up to AI interaction, including edge computing, power design, and companion robot trends.
Read more →
·5 min
Why Are China's Open-Source AI Models Getting Stronger? A Deep Dive into Three Comeback Paths
Facing US chip bans and closed-source monopolies, how do China's open-source AI models keep striking back? A deep dive into three core paths: open-source pricing-power games, optical interconnect positioning, and edge-side use cases.
Read more →
·5 min
Why Are China's Open-Source AI Models Getting Stronger? A Deep Dive Into Three Comeback Strategies
Facing US chip bans and closed-source monopoly, how do China's open-source AI models keep fighting back? A deep dive into three core paths: open-source pricing games, optical interconnect positioning, and on-device scenarios.
Read more →
·6 min
How the Creator of Redis Squeezed a 284B-Parameter DeepSeek into an Ordinary Laptop
Redis creator antirez built DS4 "Dwarf Star," a pure-C inference engine, using asymmetric quantization to compress DeepSeek V4 Flash from 500GB to 80.8GB for local 128GB unified-memory deployment at 26.7 tokens/sec.
Read more →
·6 min
How the Creator of Redis Squeezed a 284-Billion-Parameter DeepSeek Into an Ordinary Laptop
Redis author antirez built the pure-C inference engine DS4 "Dwarf Star," compressing DeepSeek V4 Flash from 500GB to 80.8GB via asymmetric quantization for local deployment on 128GB unified memory at 26.7 tokens/sec.
Read more →
·4 min
jlens-gguf: Bringing Jacobian Lens Visualization and Steering to llama.cpp Local LLMs
jlens-gguf is an open-source tool bringing Anthropic's Jacobian Lens interpretability method to GGUF and llama.cpp, enabling internal observation, real-time steering, and abliteration for both dense and MoE models.
Read more →
·6 min
AI Application Engineer Interview Guide: Core Topics in Model Quantization and Inference Deployment
A systematic review of must-know topics for AI Application Engineer interviews: PTQ/QAT quantization, operator fusion, inference pipelines, latency/throughput analysis, and edge deployment of detection/segmentation/BEV models.
Read more →