Unit testing in Totara 19

About our unit tests

Totara 19 makes use of PHPUnit 10.5 for unit testing. The installation of necessary libraries is managed via Composer. The test environment is automatically reset after each test, including database state and dataroot files.

It is important to note that since 2016 it is a requirement that every change made to code is accompanied by tests: unit tests for back-end changes, and acceptance tests for front-end changes.

Testing instructions for older Totara versions can be found at Unit testing and Unit testing in Totara 13.

Getting it running

1. Create test/phpunit/config.php

Before initialising PHPUnit you need to tell Totara how to connect to the test database and where to store test files.

You can start by copying a sample configuration file, then editing the new test/phpunit/config.php file to specify the database connection and dataroot location.

2. Initialise your site for testing

We need to initialise PHPUnit within Totara so that it is ready to run.

This will download Composer, install libraries in the vendor directory, install the database structures and prepare the dataroot directory.

You are now ready to run tests.

Run tests

This can be done in a number of ways, including but not limited to the following:

Running all tests

Running a test suite

Running a single test case

As above, except you provide a test file as an argument.

Running a single test

It's also possible to run a single test within a test case using the filter argument.

Running a group of tests

More recently we have been writing tests and defining them as groups, so that you can run related tests that are spread across the entire system with ease.

In general it is not necessary to use groups because the naming of test case files and classes makes filtering a more reliable option.

Running tests in parallel

Paratest is a tool which enables you to run the tests in parallel, reducing the overall runtime of the test suite from several hours to one hour.

We have a CI system that can run tests for you in parallel mode, and at your request against different versions of PHP and different databases.

Initiate the tests

To initiate your site for a parallel test run:

The processes parameter cannot be omitted.

Running the tests

We provide a script to trigger the run. Just supply the same 'processes' argument as during the init:

Other test options

Performance testing unit tests

To better understand the tests you are writing, and to help us understand the performance costs of our test cases, we have a means within Totara with which we can test the performance of our unit tests.

To start, add the following to your test/phpunit/config.php:

Now run tests.

You will find the following two files have appeared within your source directory:

• phpunit_profile_classes_postgres.csv

• phpunit_profile_methods_postgres.csv

These two files contain the timings on every class and method when you last ran PHPUnit.

Distribution tests

There is a special type of test that is intended for distribution purposes only. You can run them by adding the following to the test/phpunit/config.php file. These tests are run automatically on our CI, and they may fail when any third-party add-ons are present.

Isolated Tests

Tests can be run in isolation (each test running in its own PHP process) by passing along the --process-isolation flag. However, this will take much longer than a regular parallel test.

Adding new tests

Naming of test case classes and files

The test case class name must start with a component prefix (also known as frankenstyle) and must end with the _test suffix. Namespaces are not allowed.

The test case file name is derived from the class name; it omits the component prefix and uses the _test.php suffix.

For example, the server/totara/cohort/tests/enrol_test.php file contains class:

More information is available at https://docs.phpunit.de/en/10.5/installation.html.

Upgrading from previous versions

There are some notable changes from PHPUnit 9.6 that had to be changed in tests in Totara, and should be checked in your own tests.

This list is non-exhaustive, so please check the PHPUnit documentation for further deprecations and changes.

• All calls to tearDown and tearDownAfterClass must call the matching parent method.

• PHP notices/warnings/errors will not auto-convert into exceptions.
  This feature was removed from PHPUnit with no replacement, tests that rely on checking for PHP warnings were adjusted.

• Some functions in the Mocking API have been removed.
  Functions like onConsecutiveCalls have been removed. The full list can be seen in the deprecations list.

• testcase_autoloader, advanced_testcase and basic_testcase have been removed.
  The autoloader is no longer relevant as our tests must be named correctly. The other base testcase classes have been deprecated since Totara 14 and are now gone.

• Assertions assertTag and assertNotTag are deprecated.
  These still exist for some legacy tests but please do not use them on any new tests.

• Dataproviders cannot include PHP resources or closures. All objects/data returned must be serilizable.
  Isolated tests pass data between tests via the php serialize function, anything that cannot be passed through will crash tests when running in isolation.

• Data providers must return examples in the correct format.

  A collection of an array of parameters must be returned, with each entry optionally being keyed by a unique message explaining the test case. The parameters array may also be keyed, and if it is the keys must match the testcase method signature.

  // Simple Example public static function my_data_provider(): array { return [ ['propA Value', 'propB Value', 'propC Value'], // Test 1 ['propA Value', 'propB Value', 'propC Value'], // Test 2 ]; } // Named Entries public static function my_data_provider(): array { return [ 'data label 1' => ['propA Value', 'propB Value', 'propC Value'], 'data label 2' => ['propA Value', 'propB Value', 'propC Value'], ]; } // Named Properties public static function my_data_provider(): array { return [ [ 'propA' => 'propA Value', 'propB' => 'propB Value', 'propC' => 'propC Value' ], [ 'propA' => 'propA Value', 'propB' => 'propB Value', 'propC' => 'propC Value' ], ]; } // Named & Labelled public static function my_data_provider(): array { return [ 'data label 1' => [ 'propA' => 'propA Value', 'propB' => 'propB Value', 'propC' => 'propC Value' ], 'data label 2' => [ 'propA' => 'propA Value', 'propB' => 'propB Value', 'propC' => 'propC Value' ], ]; }

Known quirks

There is a visual bug when running tests in parallel with Paratest, where if a single test class has a large number of steps in it (calculated across all the test cases) then it will print some accidental output during the progress report.

This will happen three times in a normal run and is harmless.