Cascade: A Platform for Fast, Focused Edge Intelligence

Abstract: There is a growing need for IoT and other edge applications that apply artificial intelligence to streams of events, IoT data, and are optimized for realtime reactive tasks like augmented reality or gaming. Cellular 5G will bring a tsunami of such uses. Yet today’s cloud frameworks are designed to train AIs offline in a batched manner then to use pretrained models for scalable edge classification tasks. Applications that continuously acquire data and continuously respond end up fighting platform noise, and often malfunction because of inconsistent or stale data.

Cascade overcomes these issues, enabling a fast reactive edge in which AIs always see consistent, current data. Surprisingly, although consistency is normally assumed to bring cost penalties, Cascade’s approach outperforms today’s weakly consistent options, sometimes by orders of magnitude, while offering cloud-style scalability. This speedup reflects a novel design that aligns the system implementation with modern networking models and hardware. Yet there is little porting overhead: Cascade is designed to drop into modern cloud settings, enabling a high degree of legacy compatibility and excellent scalability.

Can edge AI achieve the kinds of reaction times and correctness that humans take for granted even in fast-changing settings? With Cascade, we believe that general-purpose edge intelligence is a practical reality.