Skip to content

Commit

Permalink
Add fragment on 100% test coverage
Browse files Browse the repository at this point in the history
  • Loading branch information
brandur committed Jul 2, 2023
1 parent f5f474c commit 749ead1
Show file tree
Hide file tree
Showing 2 changed files with 25 additions and 0 deletions.
6 changes: 6 additions & 0 deletions content/atoms/_meta.toml
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@
#
################################################################################

[[atoms]]
published_at = 2023-07-02T12:39:10-07:00
description = """\
Published fragment [100% test coverage](/fragments/100-percent-coverage), on whether it's a good or bad thing.
"""

[[atoms]]
published_at = 2023-06-23T12:33:04-07:00
description = """\
Expand Down
19 changes: 19 additions & 0 deletions content/fragments/100-percent-coverage.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
+++
hook = "100% test coverage is annoying as hell, but probably good."
published_at = 2023-07-02T12:39:10-07:00
title = "100% test coverage"
+++

We run one of our projects not only with 100% test coverage, but 100% _branch_ coverage. It's deeply unpleasant. Every single change you make is likely to accidentally break test coverage, so if a normal development process is **1.** add feature, and **2.** write tests/fix bugs, here you have a three-step process of **1.** add feature, **2.** write tests/fix bugs, and **3.** hunt down every omitted branch and write increasingly convoluted tests to hit them.

And I know I'm not the only skeptic. Company coverage requirements are a commonplace complaint from working developers, right up there with slow test suites and annoying management.

But 100% coverage definitely helps, especially in languages like Ruby where an untaken branch can contain a serious error that's accepted by the interpreter, but will take down production if hit.

While thinking about whether 100% coverage was an net good or ill, it occurred to me that one of its underappreciated aspects is that it's a ratchet. Another project we run is built by a small team of two and doesn't require 100% coverage. Its test suite is good, and we have an informal policy that tests are exhaustive-but-not-100%-exhaustive, but the only thing to guarantee that are good taste on the part of each developer and to a less extent, code reviews [1]. If we were to add a third person, hopefully they'd share our aesthetic preference around tests, or at least learn to emulate it, but there's no guarantee. If that third person is still chronically underwriting tests months into their tenure, undertested features will proliferate, and more regressions can be expected as changes are riskier to make.

I use the example of going from two to three people, and person number three may or may not have good testing intuition, but once you have an org of 10 or 100, you're guarantee to have someone who does not.

So as unpleasant as it is, I think I still come down on the side that 100% coverage is good. The tests themselves are useful for preventing regressions, but the more important piece might be the tight baseline for testing practice that it enforces across a larger team.

[1] Code reviews tend to catch some missing tests, but it's a huge amount of effort for a reviewer to go through and try to identity _every_ test case that might've been desirable, but omitted.

0 comments on commit 749ead1

Please sign in to comment.