Totara 15 adopts a different approach for deployment of the Recommender. The Recommender for Totara 15 runs inside an independent Machine Learning (ML) Service. The ML service is a separate module that can run on a separate server if required. Communication between Totara and the ML service is via API calls in real time. Updates to the Recommender will happen based on user activity.

In Totara 13 and 14 the Recommender communicates via a series of shell scripts as described below. This legacy Recommender is supported in Totara 15 and will continue to be fully functional when Totara 13 or 14 is upgraded to Totara 15. Totara recommends upgrading to the new Machine Learning Engine as the legacy Recommender will be deprecated in the future.

The upgrade process is described below.

Upgrading the legacy Recommender to the Machine Learning Service

This step is required only if you are using the legacy Recommender on Totara 13 or 14 and want to use the Recommender with the new Machine Learning (ML) service.

The legacy Recommender works via the execution of three separate scripts:

Export data ( php server/ml/recommender/cli/export_data.php )
Train recommender model ( eval php server/ml/recommender/cli/recommender_command.php )
Import data back to server ( server/ml/recommender/cli/import_recommendations.php )

The new Recommender does not require tasks 2 and 3. Task 1 is still required to be scheduled. If you have task 2 and 3 scheduled in cron, disable those but leave task 1 running.

All three of these tasks are included in the script server/ml/recommender/cli/run.sh. If you have run.sh scheduled, disable it and schedule export_data.php (Task 1 above). A sensible schedule for export_data.php is once every 24 hours.

Once the above steps are completed, install the new Machine Learning Service by following the installation steps below and the recommendations configurations on Totara. When the service is successfully installed, and is running and configured on Totara, the recommendations model will train in the ML Service and Totara will start getting recommendations from the Service.

Configuration

The URL of the ML Service and a secret key needs to be configured on Totara for the successful connection with the Service. This can be done from admin settings page of Totara https://your_domain/server/admin/settings.php?section=machine_learning_environment or from config.php script by adding the following lines:

$CFG->ml_service_url = 'http://mlservice:5000'; // The URL of the ML Service
$CFG->ml_service_key = 'authenticationkey';

Installation and running guide

The Machine Learning Service can be installed and run in: a Docker container, Linux (with supervisor or without), or Windows. The installation and configuration instructions on different platforms are included in integration/extensions/ml_service/README.md.

After the ML Service is successfully installed the data export script must be scheduled to run via cron:

php server/ml/recommender/cli/export_data.php

The frequency depends on how rapidly the content changes in your Totara site. Once every 24 hours is a sensible starting point.

The ML Service will fetch the data cached by this task on regular intervals. The default frequency on which the ML Service fetches the data and upgrades the recommendation model is once every 24 hours. This can be modified by following the instructions in the file: integration/extensions/ml_service/README.md

Note that the ML service needs two directories; one for ML models and one for logs. These directories can be configured for the ML Service with the environment variables ML_MODELS_DIR and ML_LOGS_DIR on the hosting platform. Users should make sure that the service has write access to these directories. The service is designed to run with all micro versions of the following minor versions of Python 3:

Python 3.6
Python 3.7
Python 3.8
Python 3.9

The ML Service can be installed and run on the same host machine as Totara (in a Docker or without), or a different Windows or Linux machine.

When configuring the ML service from scratch for Totara 15 onwards the following scheduled tasks should be disabled:

Export user data for recommendation processing (\ml_recommender\tast\export)
Import user recommendations (\ml_recommender\task\import)

They should be set up as shown below:

Two of the old recommender tasks disabled.

Note for Windows users: The Recommender in the ML Service uses a library called LightFM for modelling the recommendations. The LightFM library needs to be compiled with an OpenMP-enabled C compiler for multi-threading. As this is hard to set up on Windows, all model fitting will be single-threaded. If you’d like to use the multi-threading capabilities of LightFM on these platforms, you should try using ML Service via Docker.