Session Description

This interactive lab puts you in front of a running distributed streaming platform built on the Typelevel stack (cats-effect, fs2, http4s, smithy4s, skunk, otel4s) and asks you to diagnose what just broke.

The system is realistic: several microservices communicating over both HTTP (public-facing API) and gRPC (internal), a streaming workload backed by fs2 and Postgres, and a full observability stack (metrics, traces and logs) already wired up with otel4s and visualized with Grafana dashboards.

Everything is pre-built, pre-deployed, and runs locally with a single command. If you would rather not run anything locally, the lab will be fully available as a GitHub Codespace.

We will work through a curated series of incidents covering two flavours of production pain:

• System-wide degradation: A big chunk of the platform slows down or stops working for everyone.
• Tail-latency mysteries: most users are fine, but a small fraction see slow or failing requests.

For each incident, you will get the symptoms, access to the running system and its observability backend, and time to investigate. We will guide you through a repeatable diagnosis pipeline — metrics → traces → logs — that scales from trivial bugs to genuinely tricky ones. At least one scenario is designed so that default instrumentation is not enough: you will add your own spans and attributes with otel4s to narrow down the problem.

After each round we will walk through the diagnosis together and discuss what telemetry made visible that logs alone would not have.

By the end you will have a practical mental model for telemetry-driven debugging in Scala, hands-on familiarity with otel4s in a realistic distributed stack, and a take-home repository you can run locally to keep practising.

Setup requirements: Docker, sbt, java 17+ or use github codespace.

Nice to have (but not required):

• Comfortable working with Scala and Cats Effect.
• Familiarity with the basic ideas of distributed systems.