dotwave.ai
A technical team focused on AI inference engine research and development, dedicated to addressing cost and latency challenges in real-time AI model serving, proposing optimization approaches such as continuous inference and persistent GPU kernels.
Core Facts
Timeline (last 90 days)
The dotwave.ai team claims that the current cost of serving real-time AI models may be up to 56 times higher than it should be.
The dotwave.ai engine runs the entire model as a resident GPU program, with the CPU responsible only for feeding in new audio and retrieving output each beat.
Running all sessions expiring at the same tick as a single batch allows the model weights to be read only once for the entire group rather than once per session.
Because the work for each tick is completely identical, the compiler can fix the scheduling and memory layout before the first call (AOT compilation), eliminating runtime scheduling overhead.
On a single NVIDIA H100 SXM 80GB, the dotwave.ai engine measured 56 concurrent sessions, while the reference technology stack could only support 1.
In testing, the p99 latency per 160-millisecond tick was 147.4–147.5 milliseconds, with zero missed deadlines across three rounds totaling 84,000 session ticks measured.
Scaling from 1 to 56 sessions means a reduction of approximately 98% in GPU occupancy per call.
dotwave.ai's measurements are server-side data, excluding network and audio playback, using identical recorded inputs and approximately two minutes of context.
All data and conclusions above come from a technical note published by dotwave.ai, representing a single source that has not yet been independently reproduced by a third party.
All Facts (9)
In testing, the p99 latency per 160-millisecond tick was 147.4–147.5 milliseconds, with zero missed deadlines across three rounds totaling 84,000 session ticks measured.
65%UnverifiedThe dotwave.ai engine runs the entire model as a resident GPU program, with the CPU responsible only for feeding in new audio and retrieving output each beat.
50%UnverifiedRunning all sessions expiring at the same tick as a single batch allows the model weights to be read only once for the entire group rather than once per session.
50%UnverifiedBecause the work for each tick is completely identical, the compiler can fix the scheduling and memory layout before the first call (AOT compilation), eliminating runtime scheduling overhead.
50%UnverifiedOn a single NVIDIA H100 SXM 80GB, the dotwave.ai engine measured 56 concurrent sessions, while the reference technology stack could only support 1.
50%UnverifiedScaling from 1 to 56 sessions means a reduction of approximately 98% in GPU occupancy per call.
50%Unverifieddotwave.ai's measurements are server-side data, excluding network and audio playback, using identical recorded inputs and approximately two minutes of context.
50%UnverifiedThe dotwave.ai team claims that the current cost of serving real-time AI models may be up to 56 times higher than it should be.
50%UnverifiedAll data and conclusions above come from a technical note published by dotwave.ai, representing a single source that has not yet been independently reproduced by a third party.
50%