Keeping (large) data providers organized in PHPUnit

When using data providers for you PHPUnit tests, it’s easy to get carried away and add loads of different test cases with subtle differences in parameters. Let’s not have a discussion on whether or not your subject under test is doing too many things if you need that many test cases, let’s instead focus on how we can keep those test cases readable, understandable and manageable.
Assume say you are writing a test for the following piece of code, it sets three flags on a presentation model class depending on the current state of a session (should a primary action be shown, should a login button be shown and should an upgrade button be shown):
/**
* @param ServerRequestInterface $request
* @param PresentationModel $presentationModel
*
* @return PresentationModel
*/
public function present(ServerRequestInterface $request, PresentationModel $presentationModel): PresentationModel
{
// Set defaults
$showLogin = true;
$showUpgrade = false;
$showPrimaryButton = true;

$session = $this->sessionProvider->getSession();
// Get users loggedIn status and default to false, in case the session is fresh
$isLoggedIn = (bool)$session->get(‘loggedIn’, false);

if ($isLoggedIn) {
$showLogin = false;
$showUpgrade = true;
}

// Check if the user is logged in *and* a pro user
if ($isLoggedIn && $session->get(‘subscriptionLevel’, Subscription::LEVEL_BASIC) === Subscription::LEVEL_PRO) {
$showUpgrade = false;
$showPrimaryButton = false;
}

return $presentationModel->withVariables([
‘show_login’ => $showLogin,
‘show_upgrade’ => $showUpgrade,
‘show_primary_button’ => $showLogin || $showPrimaryButton,
]);
}

We’ll write three test cases for now:

It’s a first visit to the website and the session is still empty
It’s a visit where the logged in status is set to false and there is no subscriptionLevel set in the session
It’s a visit where the logged in status is set to true and the subscriptionLevel status is PRO.

We’ll be writing the data provider first:
/**
* @return array[]
*/
public function sessionDataProvider(): array
{
return [
[
[],
true,
false,
true,
],
[
[‘loggedIn’ => false],
true,
false,
true,
],
[
[‘loggedIn’ => true, ‘subscriptionLevel’ => Subscription::LEVEL_PRO],
false,
false,
false,
],
];
}

Would you care to guess what those undescriptive booleans mean? You’ll probably be able to figure that out, but it’ll definitely take longer than necessary. How about we make it a bit more descripive? Remember that PHPUnit will simply take the values of the arrays returned by the data provider in the order in which they are defined, it doesn’t care much about array keys.
/**
* @return array[]
*/
public function sessionDataProvider(): array
{
return [
[
‘session’ => [],
‘showLogin’ => true,
‘showUpgrade’ => false,
‘showPrimaryButton’ => true,
],
[
‘session’ => [‘loggedIn’ => false],
‘showLogin’ => true,
‘showUpgrade’ => false,
‘showPrimaryButton’ => true,
],
[
‘session’ => [‘loggedIn’ => true, ‘subscriptionLevel’ => Subscription::LEVEL_PRO],
‘showLogin’ => false,
‘showUpgrade’ => false,
‘showPrimaryButton’ => false,
],
];
}

This way it’s immediately clear what those values represent. When you’re returning to this test, six months from now, you won’t have to find the test implementation first to find the meaning of [true, false, true].
There is still room for improvement though. Even though it’s clear what the variables mean, it’s not immediately clear what we’re testing. We could do better and one way of doing it would be to provide the data set with an array key too:
/**
* @return array[]
*/
public function sessionDataProvider(): array
{
return [
‘fresh-session’ => [
‘session’ => [],
‘showLogin’ => true,
‘showUpgrade’ => false,
‘showPrimaryButton’ => true,
],
‘not-logged-in-subscription-unknown’ => [
‘session’ => [‘loggedIn’ => false],
‘showLogin’ => true,
‘showUpgrade’ => false,
‘showPrimaryButton’ => true,
],
‘logged-in-pro-user’ => [
‘session’ => [‘loggedIn’ => true, ‘subscriptionLevel’ => Subscription::LEVEL_PRO],
‘showLogin’ => false,
‘showUpgrade’ => false,
‘showPrimaryButton’ => false,
],
];
}

This again helps in the readability of your data provider. You won’t ever have to think “why did I/someone add this test case, what is it even testing?".
The data provider is now pretty readable, let’s quickly implement the test itself:
**
* @dataProvider sessionDataProvider
*
* @param array $sessionData
* @param bool $showLogin
* @param bool $showUpgrade
* @param bool $showPrimaryButton
*
* @return void
*/
public function testHeaderShouldBeShownWithCorrectButtonAction(
array $sessionData,
bool $showLogin,
bool $showUpgrade,
bool $showPrimaryButton
): void {
$this->sessionProvider->getSession()->willReturn(Session::fromData($sessionData));

$presentationModel = $this->presenter->present($this->request, $this->presentationModel);

$this->assertEquals($showLogin, $presentationModel->getVariable(‘show_login’));
$this->assertEquals($showUpgrade, $presentationModel->getVariable(‘show_upgrade’));
$this->assertEquals($showPrimaryButton, $presentationModel->getVariable(‘show_primary_button’));
}

Very straightforward, but let’s look at what the output will be if those tests fail:
There were 3 failures:

1) TestClass::testHeaderShouldBeShownWithCorrectButtonAction with data set #0 (array(), true, false, true)
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

2) TestClass::testHeaderShouldBeShownWithCorrectButtonAction with data set #1 (array(false), true, false, true)
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

3) TestClass::testHeaderShouldBeShownWithCorrectButtonAction with data set #2 (array(false, ‘basic’), true, false, true)
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

You would now have to check what data set #0 is to find out what test failed and before you can actually start debugging.
Let’s improve our setup again. Instead of using a description of the test case as an array key in the data provider, let’s make it part of the returned data set:
/**
* @return array[]
*/
public function sessionDataProvider(): array
{
return [
[
‘testCase’ => ‘Fresh Session’,
‘session’ => [],
‘showLogin’ => true,
‘showUpgrade’ => false,
‘showPrimaryButton’ => true,
],
[
‘testCase’ => ‘Not Logged In & Subscription Unknown’,
‘session’ => [‘loggedIn’ => false],
‘showLogin’ => true,
‘showUpgrade’ => false,
‘showPrimaryButton’ => true,
],
[
‘testCase’ => ‘Logged In & PRO subscription’,
‘session’ => [‘loggedIn’ => true, ‘subscriptionLevel’ => Subscription::LEVEL_PRO],
‘showLogin’ => false,
‘showUpgrade’ => false,
‘showPrimaryButton’ => false,
],
];
}

Now we can use the description of our test case in the test:
/**
* @dataProvider sessionDataProvider
*
* @param string $testCase
* @param array $sessionData
* @param bool $showLogin
* @param bool $showUpgrade
* @param bool $showPrimaryButton
*
* @return void
*/
public function testHeaderShouldBeShownWithCorrectButtonAction(
string $testCase,
array $sessionData,
bool $showLogin,
bool $showUpgrade,
bool $showPrimaryButton
): void {
$this->setName($testCase);

$this->sessionProvider->getSession()->willReturn(Session::fromData($sessionData));

$presentationModel = $this->presenter->present($this->request, $this->presentationModel);

$this->assertEquals($showLogin, $presentationModel->getVariable(‘show_login’));
$this->assertEquals($showUpgrade, $presentationModel->getVariable(‘show_upgrade’));
$this->assertEquals($showPrimaryButton, $presentationModel->getVariable(‘show_primary_button’));
}

If the tests fail now, it outputs:
There were 3 failures:

1) TestClass::Empty Session with data set #0 (‘Empty Session’, array(), true, false, true)
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

2) TestClass::Not Logged In with data set #1 (‘Not Logged In’, array(false), true, false, true)
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

3) TestClass::Not Logged In & Basic Subscription with data set #2 (‘Not Logged In & Basic Subscription’, array(false, ‘basic’), true, false, true)
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

We’re getting extremely close now, but we can step it up one more notch, because Failed asserting that false matches expected true is still not as helpful in debugging as it could be, we have multiple assertions that could expect true but are now getting false. Fortunately PHPUnit allows passing a custom error message to assertions, so we could now do this (I have removed the $this->setName() call, because we’ll get the test case description anyway and the actual test method name is now still there as a bonus):
/**
* @dataProvider sessionDataProvider
*
* @param string $testCase
* @param array $sessionData
* @param bool $showLogin
* @param bool $showUpgrade
* @param bool $showPrimaryButton
*
* @return void
*/
public function testHeaderShouldBeShownWithCorrectButtonAction(
string $testCase,
array $sessionData,
bool $showLogin,
bool $showUpgrade,
bool $showPrimaryButton
): void {
$this->sessionProvider->getSession()->willReturn(Session::fromData($sessionData));

$presentationModel = $this->presenter->present($this->request, $this->presentationModel);

$this->assertEquals(
$showLogin,
$presentationModel->getVariable(‘show_login’),
"`show_login` set incorrectly for test case \"{$testCase}\""
);
$this->assertEquals(
$showUpgrade,
$presentationModel->getVariable(‘show_upgrade’),
"`show_upgrade` set incorrectly for test case \"{$testCase}\""
);
$this->assertEquals(
$showPrimaryButton,
$presentationModel->getVariable(‘show_primary_button’),
"`show_primary_button` set incorrectly for test case \"{$testCase}\""
);
}

This would be the output if the test fails now:
There were 3 failures:

1) TestClass::testHeaderShouldBeShownWithCorrectButtonAction with data set #0 (‘Empty Session’, array(), true, false, true)
`show_login` set incorrectly for test case "Empty Session"
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

2) TestClass::testHeaderShouldBeShownWithCorrectButtonAction with data set #1 (‘Not Logged In’, array(false), true, false, true)
`show_login` set incorrectly for test case "Not Logged In"
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

3) TestClass::testHeaderShouldBeShownWithCorrectButtonAction with data set #2 (‘Not Logged In & Basic Subscription’, array(false, ‘basic’), true, false, true)
`show_login` set incorrectly for test case "Not Logged In & Basic Subscription"
Failed asserting that false matches expected true.

/path/to/TestClass.php:88

This makes it very clear at first glance:

which test failed
which test case for that test failed
which assertion in that particular scenario failed

Conclusion

Describe your test cases and test parameters in order to be able to still maintain/read/debug your test somewhere in the future. Also make sure you help your future self by providing descriptive messages when your tests/assertions fail. You’ll thank your past self when the time comes.

Link: https://dev.to//erikbooij/keeping-large-data-providers-organized-in-phpunit-983