Table-driven test design in C++ with GoogleTest

24th of April, 2022

I've always found tremendous value from testing my software. Especially what might be closest to home for developers - or at least should be - are unit tests. While unit tests are not necessarily the best way of making safe working code (this often requires little bit more exhaustive testing) but at least they're very beneficial for your future self and/or co-workers who might be working with your code since with them you can quickly see any new errors that might've come from regression.

That being said, often writing unit tests can be quite cumbersome. I would love to see some mature tooling for randomized testing like QuickCheck in Haskell (and later some other languages too) that would "just work", but often something like that just isn't possible especially when the project reaches a certain degree of complexity. Tests and test suites should be designed on their own as well as your code itself. Unfortunately, people tend to forget this. In these kind of cases quite simple table-driven test design can come to help!

I first stumbled upon table-driven test design when I was working with Go, since in there this seems to be quite popular way of doing unit tests, and at least in my opinion it works quite nicely!

Often while writing unit tests, you would want to write various failing and passing test cases, which often leads to quite a bit of duplication. For example:

Examples here are written in C++20 using Google Test.

TEST(TwoSumTests, PassingTest) {
  std::vector<int> nums{2, 7, 11, 15};
  auto got = twoSum(nums, 9);
  std::vector<int> expected{0, 1};
  EXPECT_EQ(got, expected);
}

TEST(TwoSumTests, FailingTest) {
  std::vector<int> nums{2, 7, 11, 15};
  auto got = twoSum(nums, 9);
  std::vector<int> expected{0, 123};
  EXPECT_NEQ(got, expected);
}

So even with this very simple example we can see that most of the code in the test case is duplicated and/or boilerplate. So we can do better. With quite simple table for tests we can loop through multiple tests without the need for duplication and easiness for adding new tests.

When it comes to testing functions, often we care about only what is going in and what should go out. Everything else in unit tests are often boilerplate. So where table-driven design help is setting up these input and expected outputs.

typedef struct {
  std::vector<int> nums;
  int target;
  std::vector<int> expected;
} twoSumTestCases;

TEST(TwoSumTests, BasicAssertions) {
  twoSumTestCases tests[] = {
    {
      std::vector<int>{2, 7, 11, 15},
      9,
      std::vector<int>{0, 1},
    },
    {
      std::vector<int>{3, 2, 4},
      6,
      std::vector<int>{1, 2},
    },
    {
      std::vector<int>{3, 3},
      6,
      std::vector<int>{0, 1},
    },
  };
  for (auto t : tests) {
    auto got = twoSum(t.nums, t.target);
    EXPECT_EQ(got, t.expected);
  }
}

So when we run this we can easily run all the tests at once:

[==========] Running 1 test from 1 test suite.
[----------] Global test environment set-up.
[----------] 1 test from TwoSumTests
[ RUN      ] TwoSumTests.BasicAssertions
[       OK ] TwoSumTests.BasicAssertions (0 ms)
[----------] 1 test from TwoSumTests (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 1 test.

To demonstrate failing test case, let's add new test there:

{
  std::vector<int>{3, 3},
  6,
  std::vector<int>{0, 2},
},

We get the following output:

Expected equality of these values:
  got
    Which is: { 0, 1 }
  t.expected
    Which is: { 0, 2 }
[  FAILED  ] TwoSumTests.BasicAssertions (0 ms)
[----------] 1 test from TwoSumTests (0 ms total)

[----------] Global test environment tear-down
[==========] 1 test from 1 test suite ran. (0 ms total)
[  PASSED  ] 0 tests.
[  FAILED  ] 1 test, listed below:
[  FAILED  ] TwoSumTests.BasicAssertions

 1 FAILED TEST

Extending test cases

Of course with that information, test logs can be quite misleading. Thankfully, we can just change the table to our liking. For example, we could add names to the tests:

typedef struct {
  std::string name; 
  std::vector<int> nums;
  int target;
  std::vector<int> expected;
} twoSumTestCases;

That we could then use on diagnostic messages in GTest's macros:

EXPECT_TRUE(false) << "diagnostic message"; // format to your liking

This kind of formatting we easily extend these test cases with just playing around a little bit with your test struct, so it could involve enumeration, subtests and much more. Which could help you in making fixing your tests/code easier, but also easier for adding new useful and good tests.

Tags: cpp, programming

Now playing: Justin Townes Earle - Appalachian Nightmare