5G User Plane Exposes AI Inference: IoD Makes Split Inference a Native Network Service

Mobile networks can transport tensors, but they cannot read the inference intent behind them — this is the fundamental contradiction facing edge AI as a service. A study from an rss source proposes Inference on Demand (IoD), attempting to turn split inference from a private application-layer implementation into a service that can be recognized and orchestrated on the 5G user plane.

The Problem: The Network Only Moves Data, It Doesn't Understand Inference

Current mobile systems essentially treat AI tensors as ordinary byte streams during transmission. The network can see packet size, latency, and throughput, but it cannot see "which model layer this batch of tensors belongs to, where the next hop should go, or whether it can still be split further."

This "lack of context" means previous research approaches each have their own limitations. Dynamic partitioning, collaborative inference, and split pipelines address compute allocation, but most remain at the application layer: they are either bound to specific apps, unaware of network conditions, or simply discard split semantics. As a result, the network cannot treat inference as an orchestratable service.

IoD's core judgment is: for split inference to truly become a network service, service intent, runtime partitioning, and tensor transport must all be simultaneously visible to the mobile data plane.

Split Inference is a key paradigm in edge AI: dividing a complete neural network model into multiple sub-models by layer, deploying them across end devices, edge servers, or even the cloud, with intermediate tensors transmitted over the network between the front and back segments. Its original purpose is to address limited compute/memory on devices — letting the device run only the first few layers while offloading the compute-intensive later layers to the edge or cloud. However, how to choose the granularity and boundaries of the split directly affects latency, bandwidth consumption, and overall throughput. Traditional approaches typically configure this statically at the application layer (e.g., a fixed split by layer count), lacking awareness of real-time network conditions — this is the core limitation identified in the article. Moving the split from the application layer up to the network layer means the network needs to understand the semantic ownership of tensors, not just transport byte streams.

What IoD Does: CAMARA-Style API + In-Band Tensor Context

IoD proposes an inference-on-demand abstraction with two key designs.

First, it uses a CAMARA-like API to expose split inference capabilities. CAMARA is a project in the telecom sector promoting network capability exposure, and IoD borrows this approach to let applications request inference services through standard interfaces, rather than hardcoding partition logic themselves.

Second, it carries split-aware tensor context in-band along the 5G user plane path. This means that when tensors are transmitted, accompanying partition information and inference state flow together with the data, allowing network nodes to make forwarding or compute decisions based on this without additional control plane queries.

CAMARA is an open-source project jointly promoted by GSMA and the Linux Foundation, aiming to define a standardized, developer-friendly set of RESTful APIs for telecom network capabilities (such as location, QoS, slicing, number verification, etc.), so that applications can invoke network functions without understanding underlying 3GPP details. CAMARA's value lies in abstracting operator network capabilities into cloud-service-like interfaces, lowering the integration barrier. IoD borrowing CAMARA's approach means exposing "split inference" itself as a network capability that can be requested, configured, and billed — applications only need to call an API to declare inference requirements, and the network-side orchestrator decides at which UPF and with what partitioning to execute. This provides a standardized path for operators to monetize AI inference as a value-added service and facilitates cross-vendor interoperability.

5G User Plane and UPF: In the 5G core network architecture, the control plane (CP) handles session management, policy, and mobility, while the user plane (UP) handles actual packet forwarding. UPF (User Plane Function) is the core network element of the user plane, responsible for PDR/FAR/QER rule matching, tunnel encapsulation, QoS enforcement, and traffic statistics. Programmable UPF allows custom processing logic to be inserted into the data plane (e.g., based on P4 or eBPF), which is exactly where IoD implements "in-band tensor context" — when tensor packets pass through the UPF, the node can read the partition metadata carried with the packet and decide whether to forward, cache, or execute an inference subtask locally. Compared with out-of-band control plane queries, carrying context in-band reduces signaling round trips but also increases packet payload overhead, which must be weighed under real congestion.

Testbed and Adaptive Splitting

The research team evaluated IoD on a testbed based on a programmable User Plane Function (UPF), using a partitionable visual AI workload.

The key finding of the experiment is: adaptive split selection can operate as a network-facing service while avoiding the rigid cost curve of static configuration. In other words, the network does not have to fix one partitioning scheme for all inference requests, but can dynamically adjust based on runtime conditions.

This has practical significance for operators. Static splitting means either over-provisioning resources or suffering performance collapse when load changes; adaptive splitting delegates this trade-off to programmable infrastructure for dynamic handling.

Significance: From Offloading to Network-Native AI Services

When service intent, runtime partitioning, and tensor transport are visible to the mobile data plane, split inference is no longer just "offloading computation to the edge." It becomes something the network can participate in coordinating.

This points to a path toward network-native AI services: programmable telecom infrastructure simultaneously coordinating compute, context, and connectivity to support user-facing applications. For operators searching for ways to monetize 5G capabilities, this idea of exposing inference as a network API may offer more differentiated value than simply selling bandwidth.

Of course, there is still a distance from testbed to live network. The deployment density of programmable UPFs, the standardization progress of cross-vendor APIs, and the overhead of inference context under real congestion scenarios are all questions IoD still needs to answer.

One-line Summary

IoD abstracts split inference as a network service that the 5G user plane can recognize and orchestrate, enabling adaptive partitioning and in-band tensor context.

Paragraph Summary

This article introduces Inference on Demand (IoD), proposed in a study from an rss source, aimed at resolving the fundamental contradiction that mobile networks can transport AI tensors but cannot understand inference intent. IoD exposes split inference capabilities through a CAMARA-like API and carries split-aware tensor context in-band along the 5G user plane path, enabling network nodes to directly make forwarding or compute decisions. The research team validated on a programmable UPF testbed that adaptive split selection can operate as a network service, avoiding the rigid cost curve of static configuration. This direction points to network-native AI services: programmable telecom infrastructure simultaneously coordinating compute, context, and connectivity. The article also notes deployment challenges such as programmable UPF deployment density, cross-vendor API standardization, and context overhead.