Real-Time Mode

A real-time mode of Databricks Spark Declarative Pipelines. By improving scheduling mechanisms, reducing processing overhead, and optimizing state management, it compresses the end-to-end latency of streaming pipelines to sub-second levels while retaining the ease of use of the declarative API. It is suited to operational use cases such as real-time risk control and dynamic pricing.