Scoverage: Test Coverage for the Scala Codebases
Session Abstract
You will learn about how to measure test coverage of your Scala codebase with Scoverage, the Scala 3 coverage tool. After learning the basics, we will dive deeper into how it is implemented, and talk about the hardening work done by the Scala Center to make the feature robust and reliable in interaction with the rest of the language features.
Session Description
Code coverage is a critical feature for robust quality assurance at the industry. It sounds simple: mark which lines ran. In Scala 3, Scoverage is a compiler feature: it rewrites typed trees, inserts tracking probes, and then hands the modified tree to every later compiler phase. That makes it available out-of-the-box for all Scala users with a simple compiler flag. Tight coupling with the compiler itself, though, poses a challenge in making it work with all compiler features.
This talk has a dual purpose. First, it is a guide to get started using coverage in your codebase. You will walk out of this 30-min talk having a useful skill of measuring how well your tests cover your code, thus contributing to reliability and robustness of your work.
Second, it is a technical deep dive into how coverage works under the hood, and what challenges we encountered making it work well. Coverage sits fairly early in the pipeline of compiler phases, and it modifies the trees heavily with instrumentation. Frequently, this breaks assumptions for the downstream phases that are responsible for all the numerous Scala 3 features. Examples include instrumentation interacting with erasure, singleton and wildcard types, capture checking, pattern matching, tail-recursion and infinite-loop warnings, constructor calls, parameterless methods, closures, and JVM method-size limits.
These are a lot of pieces interacting in complex ways. To make coverage work robustly, it is necessary to track, prioritize and fix breakages in a systematic way. In this talk, we will explore the process we use for that, as well as give some highlights of the typical breakages we encounter.
So the second purpose of this talk is to give a glimpse to a curious-minded engineer into the compiler internals. An industry practitioner will leave with a deep understanding not only of how to use the coverage feature, but also how it works under the hood. And for someone working on the compiler or just curious about how Scala works, it is a great opportunity to learn more about the compiler itself.
