OpenVINO
An inference optimization toolkit developed by Intel that improves deep learning model inference performance on Intel CPUs, integrated graphics, and discrete GPUs through model conversion, graph optimization, and quantization. It supports conversion from formats such as PyTorch and ONNX into the OpenVINO IR intermediate representation.
Timeline (last 90 days)
In 5.18.0, GGUF integration added support for MoE, the compressed-tensors FP8 kernel can now run on MPS (torch ≥ 2.15), an OpenVINO HF exporter was introduced, and Flash Attention support was added for XPU and MUSA platforms.
With OpenVINO and AIPC, developers can build locally running interactive applications on the CPU, GPU, and NPU of the latest AIPC hardware without users incurring token fees.
ModelBest's edge model MiniCPM 5 2B has been adapted for OpenVINO and can perform local Agent tasks such as tool calling on-device.
Intel provides the oneAPI and OpenVINO toolchains to support mainstream deep learning frameworks, but compared to the NVIDIA CUDA ecosystem, there is still a significant gap in community accumulation and framework compatibility.
OpenVINO (Open Visual Inference and Neural Network Optimization) is an inference optimization framework introduced by Intel, specifically designed to accelerate Intel's CPUs, integrated graphics, and other hardware.
A Reddit developer converted a 421-million-parameter Laya model to OpenVINO format and quantized it to INT8 precision, running a Flappy Bird game on a desktop machine equipped with an Intel Core i7 12th-generation processor.
OpenVINO's Intermediate Representation (IR) format consists of two files: .xml describing the network topology and .bin storing the weight data.
OpenVINO model conversion is typically completed via the mo (Model Optimizer) command-line tool, supporting imports from mainstream frameworks such as PyTorch (via ONNX as an intermediary), TensorFlow, and PaddlePaddle.
OpenVINO's built-in NNCF (Neural Network Compression Framework) tool supports two quantization methods: PTQ and QAT.
OpenVINO runtime's Inference Engine automatically selects the optimal execution backend based on the target hardware, invoking code paths optimized for instruction sets such as AVX-512 and AVX2 on CPUs.
8 more timeline events
All Facts (19)
In 5.18.0, GGUF integration added support for MoE, the compressed-tensors FP8 kernel can now run on MPS (torch ≥ 2.15), an OpenVINO HF exporter was introduced, and Flash Attention support was added for XPU and MUSA platforms.
50%UnverifiedWith OpenVINO and AIPC, developers can build locally running interactive applications on the CPU, GPU, and NPU of the latest AIPC hardware without users incurring token fees.
50%UnverifiedModelBest's edge model MiniCPM 5 2B has been adapted for OpenVINO and can perform local Agent tasks such as tool calling on-device.
50%UnverifiedIntel provides the oneAPI and OpenVINO toolchains to support mainstream deep learning frameworks, but compared to the NVIDIA CUDA ecosystem, there is still a significant gap in community accumulation and framework compatibility.
50%UnverifiedOpenVINO runtime's Inference Engine automatically selects the optimal execution backend based on the target hardware, invoking code paths optimized for instruction sets such as AVX-512 and AVX2 on CPUs.
50%UnverifiedOpenVINO's built-in NNCF (Neural Network Compression Framework) tool supports two quantization methods: PTQ and QAT.
50%UnverifiedOpenVINO (Open Visual Inference and Neural Network Optimization) is an inference optimization framework introduced by Intel, specifically designed to accelerate Intel's CPUs, integrated graphics, and other hardware.
50%UnverifiedOpenVINO's Intermediate Representation (IR) format consists of two files: .xml describing the network topology and .bin storing the weight data.
50%UnverifiedOpenVINO model conversion is typically completed via the mo (Model Optimizer) command-line tool, supporting imports from mainstream frameworks such as PyTorch (via ONNX as an intermediary), TensorFlow, and PaddlePaddle.
50%UnverifiedOpenVINO's GenAI pipeline (openvino-genai) adds a high-level wrapper for generative models.
50%UnverifiedOpenVINO is an inference optimization toolkit launched by Intel, aimed at achieving higher inference performance for deep learning models on Intel CPUs, integrated graphics, and discrete graphics.
50%UnverifiedOpenVINO reduces inference latency and improves throughput through model conversion, graph optimization, and quantization.
50%UnverifiedThe OpenVINO workflow is divided into two stages: offline conversion (converting PyTorch, ONNX, and other formats to OpenVINO IR) and runtime inference.
50%UnverifiedDuring OpenVINO conversion, graph optimizations such as constant folding, layer fusion, and dead code elimination are performed, with optional INT8 or FP16 quantization.
50%UnverifiedEdge AI inference typically relies on lightweight inference frameworks such as TensorFlow Lite, ONNX Runtime, and OpenVINO.
50%Unverified推理链路通过ONNX Runtime承载，后端使用Intel OpenVINO作为执行provider，并开启多设备模式和延迟优先提示
50%UnverifiedOpenVINO是英特尔开源的模型优化与部署工具链，支持将PyTorch、ONNX等格式模型自动转换并分配到最适合的计算单元运行
50%UnverifiedCPU、GPU、NPU的协同调度由英特尔的OpenVINO框架统一管理
50%UnverifiedIntel's official developer team announced Day-0-level OpenVINO support for Qwen-Image-2.1.
50%