Machine Learning Service

What is the Machine Learning Service?

The Machine Learning (ML) Service provides a space for ML and artificial intelligence (AI) enrichment within the broader context of a running Totara instance.

Explaining recommendations in the two different systems: Legacy Recommender vs Machine Learning Service

The Totara Recommendations system was introduced as part of Engage in Totara 13. The initial implementation was called Recommenders (located in the codebase under extensions/ml_recommender). This was a cron-based system that works by building a model and generating recommendations in an overnight job, which were then imported into Totara. The Recommender communicates via a series of shell scripts. While this system is relatively simple to implement, it does not scale well for large sites. Recommendations are generated every cycle rather than in real time, which means new users or new content do not have recommendations until the next cycle. 

Totara 15 adopted a different approach for the deployment of the Recommenders engine. In Totara 15 we introduced the Machine Learning Service (located in the codebase under extensions/ml_service from Totara 15 to 19) as a host for the Recommenders engine. In the ML Service, the Recommenders engine runs as a subcomponent. The recommendations are generated in real time based on a model that is refreshed periodically. This brings obvious improvements to the experience for the end user, but it also allows for better system efficiency and site performance. The ML Service is a separate module that can run on a separate host from the Totara site if required. Communication between Totara and the ML Service is via API calls in real time. Updates to the Recommender will happen based on user activity.

From Totara 20 the ML Service is no longer embedded in Totara, rather it is distributed independently via git like Mobile is. This version is designed and will still work with Totara 15 onwards.

Installing

Configuration

The URL of the ML Service and secret key need to be configured in Totara for the successful connection with the service. This can be done from via https://[your_domain]/server/admin/settings.php?section=machine_learning_environment or from the config.php script by adding the following lines:

You will also need to make sure the communication port is not blocked in Totara (defaults to 5000).

Open

This can be accessed via the Security > IP Blocker page.

Code Access

You can download the machine learning service by using git and cloning the ml repository.

$ git clone ssh://git@code.totaralms.com:totara-ml-service

If you have access to the regular Totara distribution repo, you automatically have access to the machine learning repo. If you need access please raise a support ticket with the request.

The major installation steps are documented in the service’s readme file. Follow those instructions to get the engine running.

After the ML Service is successfully installed, the data export script must be scheduled to run via cron:

The frequency depends on how rapidly the content changes in your Totara site. Once every 24 hours is a sensible starting point.

The ML Service will fetch the data cached by this task on regular intervals. The default frequency with which the ML Service fetches the data and upgrades the recommendation model is once every 24 hours. This can be modified by following the instructions in the service’s readme file.

Note that the ML Service requires two directories: one for ML models and one for logs. These directories can be configured for the ML Service with the environment variables ML_MODELS_DIR and ML_LOGS_DIR on the hosting platform. Users should make sure that the service has write access to these directories.

Upgrading between versions

From the legacy system to the latest Machine Learning service

If you’re running Totara 13 or 14, you first must upgrade to at least 15 to proceed.

If you’re running Totara 15 to 19 and have the legacy recommender system running, you can still upgrade to the latest version without changing Totara itself.

The legacy Recommender works via the execution of three separate scripts:

1. Export data ( php server/ml/recommender/cli/export_data.php ).

2. Train recommender model ( eval `eval `php server/ml/recommender/cli/recommender_comma`nd.php` ).

3. Import data back to server ( php server/ml/recommender/cli/import_recommendations.php ).

The new Machine Learning Service does not require tasks 2 and 3. Task 1 is still required to be scheduled. If you have task 2 and 3 scheduled in cron, disable those but leave task 1 running.

All three of these tasks are included in the script server/ml/recommender/cli/run.sh. If you have run.sh  scheduled, disable it and schedule export_data.php (Task 1 above). A sensible schedule for export_data.php is once every 24 hours.

Once the above steps are completed, install the new Machine Learning Service by following the installation steps above and the recommenders engine configurations on Totara. When the service is successfully installed, and is running and configured on Totara, the recommenders engine model will train in the ML Service and Totara will start getting recommendations from the service.

From Totara 15-19 Machine Learning service to Totara 20+ Machine Learning service

Shutdown any running python or docker service based on the server/extensions/ml_service codebase, then checkout and upgrade Totara’s codebase. You’ll note that the server/extensions/ml_service directory no longer exists.

Follow the instructions in the installation steps above to resume the recommendations system.

© Copyright 2022 Totara Learning Solutions. All rights reserved.