Inside Burst’s Test Interceptors

I recently posted on the Cash Code Blog announcing Burst’s new test interceptors feature. They are similar to JUnit rules in API and capability.

A Small API

A typical use case is connecting some lifecycle to a test’s execution. Perhaps we want a chess engine to be available while a test is running:

class ChessEngineTest {
  @InterceptTest
  val chessEngineInterceptor = ChessEngineInterceptor()
  
  val chessEngine: ChessEngine
    get() = chessEngineInterceptor.chessEngine

  @Test
  fun moveValidation() {
    assertThat(chessEngine.isLegalMove(E2, E4)).isTrue
  }
}

Writing an interceptor is straightforward:

class ChessEngineInterceptor : TestInterceptor {
  lateinit var chessEngine: ChessEngine

  override fun intercept(testFunction: TestFunction) {
    stockfish = StockfishChessEngine()
    try {
      testFunction()
    } finally {
      stockfish.close()
    }
  }
}

Critically, the interceptor calls testFunction() to execute the test body. This gives it lots of power:

• It can differentiate test successes (testFunction() returns normally) from test failures (testFunction() throws)
• It can skip tests by not calling testFunction()
• It can repeat tests by calling testFunction() in a loop
• It can collect timing metrics by calling testFunction() inside a measureTime() block

JUnit rules have the same capabilities. JUnit rules are implemented with Java reflection, which is a great fit for both simplicity and capability.

One Possible Implementation

But Burst implements the same behavior without reflection. Burst is a Kotlin Multiplatform library and uses a  compiler plugin to connect interceptors to the tests they execute.

Here’s a sketch of how Burst could have transformed ChessEngineTest to hook up its single interceptor:

class ChessEngineTest {
  @InterceptTest
  val chessEngineInterceptor = ChessEngineInterceptor()

  val chessEngine: ChessEngine
    get() = chessEngineInterceptor.chessEngine

  @Test
  fun moveValidation() {
    chessEngineInterceptor.intercept(
      object : TestFunction(
        packageName = "com.publicobject.chess",
        className = "ChessEngineTest",
        functionName = "moveValidation",
      ) {
        override fun invoke() {
          assertThat(chessEngine.isLegalMove(E2, E4)).isTrue
        }
      }
    )
  }
}

But this isn’t how Burst does it...

Implementation Challenges

One test function and one interceptor is no sweat! But we’re building Burst to power a particularly thorough set of tests that have more requirements.

Multiple interceptors. We want these to execute in declaration order.

Inheritance. We can have interceptors in the base class, and in each subclass. When a test function runs we want to execute all the interceptors. Different subclasses could declare different interceptors!

abstract class AbstractChessEngineTest {
  @InterceptTest
  abstract val engineInterceptor: ChessEngineInterceptor

  val chessEngine: ChessEngine
    get() = engineInterceptor.chessEngine

  @Test
  fun moveValidation() {
    assertThat(chessEngine.isLegalMove(E2, E4)).isTrue
  }
}

class StockfishChessEngineTest : AbstractChessEngineTest() {
  override val engineInterceptor = StockfishEngineInterceptor()
}

class GenerativeAiChessEngineTest : AbstractChessEngineTest() {
  override val engineInterceptor = StockfishEngineInterceptor()

  @InterceptTest
  private val flakyTestInterceptor = FlakyTestInterceptor()
}

In this example, moveValidation() runs with one interceptor in StockfishChessEngineTest, and two in GenerativeAiChessEngineTest.

Module isolation. Each file in our test’s class hierarchy can be compiled independently. We can introduce a new interceptor in the base class without recompiling the subclass, or vice-versa.

The Actual Implementation

Each test that defines interceptors aggregates those interceptors up into its own intercept() function, that calls them in the appropriate order.

If the superclass has interceptors, we call its intercept first, passing the subclass stuff as an argument.

The net effect is that we get to apply a chain-of-responsibility pattern for each interceptor in a test, and apply that pattern again for each class in the type hierarchy.

Check it out

I had a lot of fun finding a way to build this thing that’d have ‘reflection-like behavior’ without any reflection.

Get Burst on GitHub.