PUBLIC OBJECT

Finding, Killing Flaky Tests

Flaky tests are the worst. They break the continuous build, the presubmit checks and confidence in the test suite.

Fixing flaky tests is unpleasant because you need a way to reliably reproduce something unreliable. And a way to confirm that you've fixed it. I prefer brute force:

  @Test public void pushPromiseStream() {
    // This test is flaky.
    ...
  }

@Test public void pushPromiseStream1000x() {
    // This test fails quite reliably!
    for (int i = 0; i < 1000; i++) {
      System.out.println(i);
      SpdyConnectionTest test = new SpdyConnectionTest();
      test.setUp();
      test.pushPromiseStream();
      test.tearDown();
    }
  }

The looping test finds the problem. I fix it. Then I can delete the looping test which has served its purpose.

There's probably better ways to do this, but this approach works for me.