Day 3 -

Sept 14th

Stage 3 (Plato 3)

15:30 - 16:15

Crossing the Boundaries of Stateful Streaming and Actors Using Serverless Portals

We present the Portals Project, our envisioned future of dataflow streaming systems with native support for serverless computing. Dataflow streaming has been widely adopted, via systems like Spark and Flink, due to its capability to process large amounts of data in real-time. However, writing certain applications in the dataflow streaming model can be difficult.

Portals is a completely new programming model that introduces novel abstractions, called Atomic Streams and Portals, that simplify building applications where multiple dataflow streaming pipelines need to communicate with each other. At the same time, Atomic Streams ensure that exactly-once processing is guaranteed end-to-end, even across multiple stateful dataflow pipelines. These additions enable writing new applications that were impossible to express previously in the dataflow streaming model: microservices such as shopping carts; iterative programming models such as the actor model or the bulk synchronous parallel (BSP) model, and streaming analytics applications.

This talk will provide an overview of the Portals model, demo several examples, including a fault-tolerant actor model, and present the open-source implementation and project, inviting contributions from the Scala community.

Jonas Spenger

KTH Royal Institute of Technology / RISE Research Institutes of Sweden

Jonas Spenger is a PhD Student at KTH Royal Institute of Technology / RISE Research Institutes of Sweden in Stockholm, Sweden. His main interests are the design and implementation of distributed systems. He is one of the core members of the Portals Project. Previously, Jonas worked on various research projects in the Berlin area, where he also received his BSc and MSc in computer science at HU Berlin.

Subscribe

Join our conference

Subscribe and follow @ScalaDays on Twitter for the latest conference updates.