TinyML
An extreme form of edge AI focused on running machine learning inference on microcontrollers (MCUs) with only tens of kilobytes of RAM and severely constrained compute resources. Representative frameworks include TensorFlow Lite Micro and Edge Impulse.
Timeline (last 90 days)
Split inference addresses the problem of insufficient hardware resources on a single device by deploying a TinyML model in segments across multiple IoT devices.
TinyML devices are constrained by SRAM (often only a few hundred KB) and flash memory (a few MB), making it difficult for a single device to host a complete model.
Replicas themselves occupy already scarce device storage resources, and on extremely resource-constrained TinyML devices, whether replicas can fit and how many replicas can fit may be a more prior question than latency optimization.
TinyML compresses models to run on MCUs with KB-level memory and milliwatt-level power consumption.
TinyML模型几乎都是单一任务、固定功能的，通常只有几十KB到几MB大小，而Needle定位为具备通用能力的基础模型
All Facts (6)
Split inference addresses the problem of insufficient hardware resources on a single device by deploying a TinyML model in segments across multiple IoT devices.
50%UnverifiedTinyML devices are constrained by SRAM (often only a few hundred KB) and flash memory (a few MB), making it difficult for a single device to host a complete model.
50%UnverifiedReplicas themselves occupy already scarce device storage resources, and on extremely resource-constrained TinyML devices, whether replicas can fit and how many replicas can fit may be a more prior question than latency optimization.
50%UnverifiedTinyML compresses models to run on MCUs with KB-level memory and milliwatt-level power consumption.
50%UnverifiedTinyML模型几乎都是单一任务、固定功能的，通常只有几十KB到几MB大小，而Needle定位为具备通用能力的基础模型
50%Unverified典型微控制器的RAM仅有数十KB，主频不超过数百MHz
50%