/
Code coverage

Code coverage

Overview

Code coverage is a valuable tool to identify gaps in your test suite and ensure that your code is adequately covered by tests. However, high code coverage alone does not guarantee high-quality tests. It is critical to focus on writing well-constructed tests, as this will have a greater impact than striving to cover every line of code.

PHPUnit and code coverage

PHPUnit provides built-in support for generating code coverage reports in various formats. To generate code coverage, PHPUnit relies on either XDebug or pcov. While pcov is more performant than XDebug, this guide demonstrates how to use both.

Code Coverage via Totara Docker Dev

If you are developing using our docker-dev environment, then code coverage is configured for you out of the box, with some helper commands to help you configure, run and generate code coverage reports.

See the documentation here on how to get started.

 

Manually Configuring Code Coverage

Configuring and generating code coverage reports (latest version)

This guide details how to configure and generate a code coverage report for the latest version of Totara. For older implementations, please see the Configurations for older versions section.

Step 1: Initialise PHPUnit

php test/phpunit/phpunit.php init

Step 2: Configure whitelists

To generate code coverage, you need to define a whitelist in your phpunit.xml file. Without this, you will encounter the following error:

Error: Incorrect whitelist config, no code coverage will be generated.

Add the following whitelist configuration to your phpunit.xml file directly after the closing </php> tag. Adjust the paths to match your project’s structure. Only directories and files included in the whitelist will appear in the coverage report.

These are just examples. Please change the whitelist so it matches your needs. Only directories and files included in the whitelist will appear in the report.

File: test/phpunit/phpunit.xml

</php> ... <source> <include> <directory suffix=".php">/var/www/totara/src/SITENAME/server/totara/core/classes/local</directory> <directory suffix=".php">/var/www/totara/src/SITENAME/server/totara/core/classes/task</directory> <file>/var/www/totara/src/SITENAME/server/totara/core/classes/visibility_adviser.php</file> </include> </source>

Important notes

  • Use full paths to avoid incorrect whitelist configuration errors.

  • You can also use <exclude> tags. Please refer to the PHPUnit documentation for the full configuration details.

  • Limit the whitelist to relevant files and directories to reduce the report generation time.

  • If <source> does not work, replace it with <coverage>.

Step 3: Generate coverage reports

Using XDebug

Ensure XDebug is installed. Use the following commands to generate an HTML-formatted coverage report:

XDEBUG_MODE=coverage php test/phpunit/phpunit.php run --filter=totara_core --coverage-html /var/www/html/coverage_report

Clover XML Report

XDEBUG_MODE=coverage php test/phpunit/phpunit.php run --filter=totara_core --coverage-clover /tmp/coverage_report.xml

Using pcov

pcov is a high-performance alternative to XDebug. To set it up:

  1. Install pcov and note the generated path to pcov.so, e.g. /usr/local/lib/php/extensions/no-debug-non-zts-20230831/pcov.so:

    pecl install pcov

     

  2. Update php.ini. Note here also that we update the memory_limit to 8G:

    memory_limit=8G [pcov] extension="path/to/pcov.so" pcov.enabled=1 pcov.exclude='~(vendor|tests|node_modules|.git|client|.scannerwork)~' pcov.initial.memory=1073741824 pcov.initial.files=30000

To find your php.ini’s location, run the following command and see the path in the output:

php --ini

  1. Generate the report:

    phpunit --coverage-html /var/www/html/coverage_report /path/to/plugin/to/test

  2. The coverage report will be generated to /var/www/html/coverage_report, which you can configure your webserver to view it.

Configurations for older versions

Totara 13

Initialisation:

php test/phpunit/phpunit.php init

Whitelist configuration:

These are just examples. Please change the whitelist so it matches your needs. Only directories and files included in the whitelist will appear in the report.

<filter> <whitelist> <directory suffix=".php">/var/www/totara/src/SITENAME/server/totara/core/classes/local</directory> <directory suffix=".php">/var/www/totara/src/SITENAME/server/totara/core/classes/task</directory> <file>/var/www/totara/src/SITENAME/server/totara/core/classes/visibility_adviser.php</file> </whitelist> </filter>

Make sure you use full paths to the files and directories. Otherwise PHPUnit might show an error about an incorrect whitelist configuration.

Generating coverage reports:

XDEBUG_MODE=coverage php test/phpunit/phpunit.php run --filter=totara_core --coverage-html /var/www/totara/src/dev/coverage

Totara 12

Initialisation:

php admin/tool/phpunit/cli/init.php

Whitelist configuration:

These are just examples. Please change the whitelist so it matches your needs. Only directories and files included in the whitelist will appear in the report.

<filter> <whitelist processUncoveredFilesFromWhitelist="false" addUncoveredFilesFromWhitelist="false"> <directory suffix=".php">/var/www/totara/src/SITENAME/totara/core/classes/local</directory> <directory suffix=".php">/var/www/totara/src/SITENAME/totara/core/classes/task</directory> <file>/var/www/totara/src/SITENAME/totara/core/classes/visibility_adviser.php</file> </whitelist> </filter>

Make sure you use full paths to the files and directories. Otherwise PHPUnit might show an error about an incorrect whitelist configuration.

Generating coverage reports:

XDEBUG_MODE=coverage vendor/bin/phpunit --filter=totara_core --coverage-html /var/www/html/coverage_report

pcov installation:

Install pcov/clobber which adds pcov support to PHPUnit 7:

composer require pcov/clobber vendor/bin/pcov clobber

Integration with PHPStorm

PHPStorm has the ability to run code coverage analysis in product using XDebug or pcov. The analysis is automatically loaded into the platform and displayed both in the coverage tool, and in-editor when looking at files. Their documentation on code coverage explains how to set it up.

If you are using docker-dev, follow the steps here to run coverage within PHPStorm.

Here are a few pointers and tips:

  • You still need to define a whitelist

  • As a product we support several different versions of PHP - if you're using php-fpm and have them installed you can configure PHPStorm so that you can choose which version you run tests and coverage on

  • When looking at a testcase file you can click on the little arrows next to test case classes and test case methods and choose to run specific tests

Common pitfalls

  1. Incorrect phpunit.xml paths: Ensure paths point to the correct Totara installation.

  2. Whitelist misconfiguration: Use pwd to verify paths and ensure full paths in your configuration.

Further reading