High level architecture and organisation

Owned by Sam Hemelryk
Oct 17, 2019

Totara has been in development for more than 9 years, and is based upon Moodle which has been in development for more than 17 years.
In terms of code all manner of best practice and patterns can be found within the code base.
The coding style for the product adapts as best practices are formed, as new technologies are adopted, and as better patterns emerge.
Retrospectively code that is stable is not refactored unless there is reason to do so.

However a high level architecture still runs through the product and is what is described below.


Organisation of structure

Totara organises its code base into several types, each of which has a distinct place in the product.

Core

The engine of the product, it underpins everything.
Each of the functions that the core serves are indispensable.
The following are examples of just some of those functions:

  • Environment controls
  • Database definition
  • Database manipulation
  • Installation and upgrade management
  • Configuration management
  • Session control
  • Language and localisation systems

Included in the core system is the management of component, plugin types, plugins, and subplugins.
Extensibility is exposed through plugin types, and subplugin types.

Components

Also referred to as subsystems, these add essential functionality to the product.
Each of these introduce stand-alone functionality. They may introduce functionality to be used by other components plugins and subplugins, or introduce functionality the user experiences, sometimes both.
There are over 50 components within the current latest release of Totara.
The following are some examples of components within Totara:

  • Backup and restore
  • Cache system
  • Completion
  • Form libraries
  • Messaging
  • Product administration
  • Repositories

Components are defined in core, and cannot be introduced by any other system. The list of components is fixed by the product.
They are no immediately extensible, however they may choose to expose plugin types that relate specifically to them.
A good example of this is the cache system which exposes the cache store and cache lock system plugin types.

Plugin types

There are 32 plugin types within the current latest release of Totara, each one allows for Totara, partners, subscribers and third parties to extend the functionality of the product.
Very importantly it is through the plugin types that Totara introduces most of the functionality that is available out of the box.

The following are examples of just some of the plugin types that are available:

  • Activities
  • Administration tools
  • Authentication methods
  • Blocks
  • Course formats
  • Enrolment methods
  • Filters
  • Themes

Plugins

A plugin introduces specific functionality to the system.

  • The LDAP authentication plugin allows users to authenticate against an organisations LDAP server.
  • The Forum module allows a forum to be added to a course, facilitates users starting threads and posting replies.

Much of the functionality available within Totara is encapsulated as plugins.
Plugins may be self serving and operate independently of all other plugins, or they may declare dependencies and build upon the functionality provided by other plugins.

Out of the box Totara ships with over 600 plugins installed, and there are thousands of plugins available in the ecosystem.
Plugins can be installed at any time in the product by putting them into the correct plugintype directory,  many can be uninstalled and removed from the code base through managed channels.

It is common to see partners installing sets of preferred plugins.

Subplugin types

Each plugin can choose to introduce subplugin types if they so choose to.
These work just like the plugin types discussed above, except that they can be introduced by third parties and the purpose of them (and the functionality they can introduce) is defined by and limited to the scope of the plugin introducing them.

Subplugins cannot introduce further plugin types, sub plugins are as deep as the system allows.

The following are some examples of sub plugins within the system:

  • The assignment activity introduces the submission subplugin type so that third parties can introduce their own assignment submission methods.
  • The database activity introduces the field subplugin type so that third parties can introduce their own field types.
  • The Atto editor introduces a plugin subplugin type so that third parties can introduce their own buttons and extensions for the editor.

Within our product we try to minimise the use of subplugins, and when adding new subplugin types we always try to ensure that the extensibility they introduce is as limited in scope as possible.

Subplugins

Just like plugins these introduce specific functionality, however they do so within a plugin.
The purpose is typically much more restricted and the integration much smaller.


Pluggable and extensible

The above describes the structured organisation of code implemented by Totara and to a large extent the pluggable architecture desribed above facilitates a means of providing functionality.
In addition their is the concept of extensibility. Often functionality can only be complete when other plugins, components, or core, can add to it, and extend it.
In some situations plugins may only exist to facilitate this level of extension, particularly when viewed from the down stream perspective where they enable extension without requiring changes to the code provided by Totara.

We have several means of allowing plugins and subplugins to extend and customise the functionality and experience of the platform.
These systems allow extension of the product and its functionality in very diverse ways:

  • The logging system within Totara has an event observer that listens to all events triggered within the product, and logs the component/plugin, event name, date+time, users involved, and any other relevant information the event is holding onto.
  • Report builder looks for the existence of classes in a dedicated directory structure that also extend the report base class. These classes are report builder report sources, and can used by the user to create custom reports. This allows plugins and subplugins to introduce their own sources and reporting into Totara report engine.
  • The main menu provides the administrator with quick access options to customise the menu. The options that are available are found by looking for autoloaded classes existing in a reserved namespace and extending the menu item interface. This allows plugins and subplugins to introduce their own menu options to make it easy for site administrators to customise and tailor the navigation experience of their learners.
  • The navigation engine checks for functions existing in a plugin or subplugins lib.php file, that match a certain pattern {pluginname}_extends_navigation() and if found calls those plugins at the end of its generation enabling them to modify the navigation tree that drivers the user experience.

Each means of extensibility has an intent and purpose, as described below:

Events

Events are our primary means of extensibility.
They serve our logging system, and are prolific throughout the product and many of the plugins we are aware of in the product eco system.
Moodle uses the same event system, as such plugins developed for Moodle that fire events continue to do so in Totara.

Events should be fired for actions occurring within the system. This is ultimately because events get logged by the logging system and provide with ease provide auditability for the platform.
If its worth logging an event should be triggered, allowing the logging system to record it, and allowing other plugins and subplugins to react to the event.

Hooks

Hooks are unique to Totara. They are designed to be light weight and serve the purpose of enabling plugins and subplugins to react to events in code flow such as the definition of a specific form being completed, the form data being ready for use, the page header having been rendered, or the pre-filling of a cache having been completed.
These events don't warrant logging, they are just things that occur in the flow of code, but that plugins and subplugins want want to react to.
For instance the definition of a form being completed would enable a plugin to modify the rules of the form, or modify the definition itself. Likewise the hook when the form was being processed would enable the plugin to respond to its earlier manipulations.

Hooks are very rarely observed in core code. In fact you could go so far as to say that they should not be observed in core code unless there is a very good reason to do so.
They exist to serve the purpose of third party plugins and subplugins, as a means to introduce extensibility in situations where core modifications would otherwise be required.

Structure existence

This is most often encountered when looking at functionality that is driven by integrations. The two examples above, report builder and the main menu are two prime examples.
Report Builder allows the user to create custom reports within Totara. It operates directly on the database, and thus is reliant upon a definition of the data and how it relates existing in code.
Each plugin and subplugin can have its own data structure, and as such it is up to each plugin and subplugin to provide that definition. Where it chooses to.
They do this by defining Report Builder sources. These sources are expected to exist within a specific directory within the plugins root directory, and are expected to extend a base implementation of a source.
Report Builder looks in that specific directory of every plugin (if it exists) and for each file it finds checks if it contains a class that extends the expected type. If it does then that class is loaded into the report builder engine as a source that the user can choose to build their report from.

Report builder is actually an old example, from the days before autoloading and namespacing.

These days we prefer to use autoloaded classes, and the namespace of the class (mapped to the location of the class in the autoloading system) answers the question of existence and intent.

Finally back in the very early days of Totara (>8 years) there was a pattern through which functions were expected to exist in specific files, named in specific ways, to facilitate this kind of extension.
This was required because none of classes, namespaces, or autoloading were available.
You will still find a few of these around, however we do not introduce more. We prefer either events, hooks, or autoloaded namespaced typed classes.


Interacting with Totara from within a plugin

This warrants a small explanation in light of architecture as it will tie together the last section on extensibility and the next section on typical component, plugin or subplugin architecture.

Once a plugin or subplugin is executing extensible code it will often need to interact with core, or its components.

This is done via in-code public API's.

The easiest way to explain this is to take a practical example from the system.

All entry pages include config.php, which internally includes the base libraries and static components which we will depend upon.
Including (but of course not limited to) the database.
Totara is database dependent, and more than half of all plugins in the system have a data structure and use the database.
In code the database is utilised by interacting with an instance of a database specific class extending moodle_database.
This instance is stored in a globally accessible variable $DB. Once declared global within your code structure you can interact with the $DB object using the DML API.
This is the in-code public database API.

There are several patterns for in-code public API within Totara:

  • Global instances as described in the example above.
    There are five in total:
    • $DB - the database.
    • $CFG - the site configuration.
    • $USER - the currently logged in user.
    • $SESSION - the users session (not to be used in normal code)
    • $SITE - the site course (not to be used in normal code)
  • Public classes either included during initialisation (config.php inclusion) or via autoloading.
    Documentation below explains how these classes are structured and organised.
  • Functions within a plugins lib.php file
    These must be named carefully, beginning with the plugins frankenstyle name.
    A full explanation of this can be found below.

Our preference is always towards autoloaded classes within core or its components.
Plugins and subplugins should not ever present an in-code public API. Interaction between plugins should always happen view the means of extension explained above.


Typical component, plugin, or subplugin architecture

As discussed at the very start of the document what can be found in code varies greatly, however there is a typical architecture that nearly all components, plugins and subplugins follow.

There are many different works parts to our architecture, and while we try to stick to this you will find abuses if you look hard enough.
Sometimes because of non-standard requirements, sometimes because of early work and inexperienced developers.

Ultimately it all must be considered from the organisation of structure explained above.
Our product provides functionality, and that functionality is encapsulated into one of core, component, plugin or subplugin.

The diagram above tries to visualise the structure of our architecture at the component, plugins, and subplugin level.
However the longer you look at it the more you realise it is incomplete.