Edge AI

Local deployment can compress AI response latency from hundreds of milliseconds to single-digit milliseconds, while preventing user data from leaving the device.

On-device AI refers to completing the inference process of machine learning models directly on the end device rather than uploading to cloud servers for processing, requiring model compression and hardware acceleration to compensate for the computational gap.

On-device inference can reduce sign language translation latency from 200–500 milliseconds in cloud-based solutions to under 50 milliseconds.

边缘AI（Edge AI）是将人工智能计算从云端迁移到本地设备（手机、传感器、嵌入式系统）上运行的技术范式

Edge AI reduces reliance on centralized cloud infrastructure; when networks are interrupted or cloud services are unavailable, edge nodes can still independently run critical tasks, improving system resilience.

A white paper published by IEEE Spectrum points out that the key to the leap from smart cities to cognitive cities lies in whether infrastructure can complete sensing, reasoning, and action under real-time conditions, and that edge AI is the technological foundation of this leap.

Edge AI has three structural advantages: low-latency decisions, data localization, and reduced reliance on the cloud.

By completing recognition and response at edge nodes, edge AI can achieve low-latency decisions at the millisecond level without waiting for data to travel to and from the cloud.

Edge AI keeps raw data within city boundaries and completes processing only at edge nodes, which can reduce data leakage risks and address compliance requirements.

Edge scenarios typically choose small models with 1B to 3B parameters, where a 3B model is about 1.5GB after 4-bit quantization and a 1B model is about 500MB.

A static-threshold model running in parallel as a safety monitor can veto adaptations and trigger alarms, and this hybrid scheme (adaptive model for fine-grained state estimation, frozen model for life-safety alarms) proved more robust in testing.

For products such as children's toys that involve data of vulnerable groups, local processing should be a precondition for whether the product can go on the market, rather than a nice-to-have feature.

Consumer AI toys like Eiliko generally adopt a 'thin client + cloud large model' architecture, where the device itself is only responsible for picking up sound, playback and expression driving, while speech recognition (ASR), natural language understanding (NLU) and speech synthesis (TTS) are all done on remote servers.

The 'thin client + cloud large model' architecture can reduce per-unit hardware costs and support rapid model iteration, but the cost is that every sentence of conversation must leave the device.

Edge AI solutions deploy miniaturized models on local chips, so data does not leave the device; limited by compute power and power consumption, their conversational ability is usually weaker than cloud solutions.

The economics of small models in edge scenarios flip: a one-time silicon cost replaces the per-token API bill.

On-board inference enables drones to independently complete the perception-decision-action closed loop in contested environments with GPS denial, communication jamming, or signal blockade, without needing to transmit images back to the cloud or a ground station.

Edge AI inference typically completes preliminary recognition on the device side, then uploads structured data to the cloud to generate historical reports.

On edge devices, due to resource constraints, speech recognition, model inference, and speech synthesis often require serial pipeline scheduling rather than parallel execution, which increases end-to-end response latency.

The return on investment cycle for deploying AI on edge devices is typically 3–12 months.