Code coverage

Owned by Fabian D.
Last updated: Aug 01, 2024
3 min read

Code coverage is a great tool to find gaps in your code coverage and making sure your code is covered with tests. High code coverage does not guarantee great test quality so please make sure the tests are well written. At the end this will likely be more important than to cover every line of code.

PHPUnit and code coverage

Traditionally PHPUnit comes with support for generating code coverage reports in different formats.

Over time PHPUnit changed how it is being configured and run. As different Totara versions also come with different PHPUnit versions this document outlines the approach to set up your environment and generate coverage for multiple versions.

To generate code coverage PHPUnit relies on either XDdebug or pcov. The performance of pcov is superior to XDebug but we will show how you can run it with both.

How to configure and generate code coverage reports

Initialise PHPUnit

The first step is to initialise the PHPUnit environment.

Totara 13 and above

php test/phpunit/phpunit.php init

Totara 12

php admin/tool/phpunit/cli/init.php

Whitelists

In order to generate code coverage you need to provide a whitelist in the phpunit.xml file otherwise you get an error when trying to generate the report.

Error: Incorrect whitelist config, no code coverage will be generated.

Add the following to the phpunit.xml file directly after the </php>tag.

Totara 14 and higher

<coverage> <include> <directory suffix=".php">/var/www/html/server/totara/core/classes/local</directory> <directory suffix=".php">/var/www/html/server/totara/core/classes/task</directory> <file>/var/www/html/server/totara/core/classes/visibility_adviser.php</file> </include> </coverage>

Totara 13

Totara 12

Same as for Totara 13 except that you have to add the processUncoveredFilesFromWhitelist="false" addUncoveredFilesFromWhitelist="false" attributes.

This is just an example. Please change the whitelist so it matches your needs. Only directories and files included in the whitelist will be showing up in the report.

Please make sure you use full paths to the files and directories. Otherwise PHPUnit might show an error about an incorrect whitelist configuration.

You can also use <exclude> tags. Please refer to the PHPUnit documentation for the full configuration details.

The more files are included in the whitelist the longer it takes to generate the coverage report. It is advised to limit it to the files / directories you are interested in.

Generating coverage

XDebug

Please make sure you have xdebug installed.

To generate the coverage report use the following command to generate a HTML formatted report.

Totara 13 and above

Totara 12

Please note that this only runs the test for a particular area of the code to reduce the runtime. This should match what you have configured as a whitelist.

You can also generate a Clover XML report to be used in other tools:

Totara 13 and above

Totara 12

Please refer to the PHPUnit documentation for the available formats.

pcov

pcov is an alternative to XDebug. It is much more performant.

To get it running:

  1. Install the pcov extension with

  2. Only for Totara 12: Install pcov/clobber which adds pcov support to PHPUnit 7

  3. Enable and configure the extension in the php.ini

  4. Ensure PHP has enough memory in php.ini. Give it heaps, as code coverage is memory intensive.

  5. Generate the code coverage

    Totara 13 and above

    Totara 12

This will create a nicely formatted report for you to inspect.

  2. Generate the code coverage

PHPStorm and XDebug/pcov

PHPStorm has the ability to run code coverage anaylsis in product using XDebug or pcov. The analysis is automatically loaded into the platform and displayed both in the Coverage tool, and in-editor when looking at files. Their documentation on code coverage explains how to set it up.

A few pointers and tips to help however:

  • You still need to define a whitelist

  • As a product we support several different versions of PHP - if you're using php-fpm and have them installed you can configure PHPStorm so that you can choose which version you run tests and coverage on

  • When looking at a testcase file you can click on the little arrows next to test case classes and test case methods and choose to run specific tests

Further reading