Totara 13 changes to content sanitisation

Several changes were made to the sanitisation of content and files in Totara 13.
These changes together make the product more secure out of the box.

Changes made in Totara 13

The following documents the changes made in Totara 13 in order to provide better out of the box protection to XSS attacks.

A new setting "Disable consistent cleaning" has been introduced

Several of the changes made in this release remove what was once possible in the name of providing a more secure product out of the box to all subscribers.

The changes reflect the growing security awareness of organisations in the modern web world.

However it does remove what was previously possible.

Privileged users could add advanced HTML such as iFrames, JavaScript and objects to content that they were currating in order to build the experience that they wanted their end users to have. An element of trust has always been required. Each capability associated with the ability to add advanced HTML is marked with an XSS risk, and can be reported upon within the product.

While we feel that the new default behaviour is the right decision for the future, and a more secure default, we have added a new setting Disable consistent cleaning that when turned on will revert the site back to the old behaviour. Users who hold the required capabilities will once again be able to embed iFrames, JS, and objects.

We strongly recommend you leave this setting off. This will ensure that there is much less risk of users introducing vulnerabilities through content on the site as all user generated content will be cleaned and passed through HTML purifier before being displayed to the end user.

This setting is now reviewed and shown on the Security overview report.

Changes made by TL-20729.

The noclean option is now ignored by default

Dozens of areas within Totara previously used to allow users who had the ability to create and edit content to also embed advanced HTML such iFrames, JavaScript, and object tags into the content. In these areas content was not consistently sanitised, and consequently the advanced HTML was displayed to the end user and could be used to introduce security vulnerabilities through either malicious intent or ignorance. This was facilitated in these areas by passing a "noclean" option to the functions responsible for cleaning and sanitising user created content.

As of Totara 13 the "noclean" option is now ignored and has no effect by default. Consequently all user created content is now consistently sanitised before display to the end user. This to ensures it does not contain any security vulnerabilities.

If the Disable consistent cleaning setting is turned on then the previous behaviour will be restored.

Changes made by TL-20729.

Trusttext is now forcibly disabled by default

When trusttext is turned on, users who hold the trust capability, and are editing content that is "trustable", will be able to embed advanced content. This system was used only in a handful of places.

As of Totara 13 the trusttext system is forcibly turned off and disabled. The setting to control it, and the capability will not be visible within the system. Any sites that had turned this on, and given trust, will find that trusted users can no longer embed advanced content.

The default behaviour now is that all content is sanitised prior to display to the user. There is no replacement for the trusttext system, however if the Disable consistent cleaning" setting is turned on then the previous behaviour will be restored.

Changes made by TL-20729.

Content-disposition when serving files now defaults to attachment

All files mediated by Totara are served with a content-disposition header. As of Totara 13 this will be forced to "attachment" by default, with only whitelisted mimetypes being allowed to use the "inline" attachment. 

A notable change is that SVG images embedded in content will not be served with an inline content disposition header. The browser will still display the image inline, however if you attempt to open the image in a new tab or view the image directly you will be prompted to download the SVG image instead of the browser displaying it.

Some areas, such as the SCORM module, are known to serve files that must be delivered with the with an inline content disposition. These areas must now influence the the file serving to use the "inline" content disposition.

If the Disable consistent cleaning setting is turned on then the previous behaviour will be restored.

Changes made by TL-20729.

New Allow XSS setting for page and label modules

The default behaviour now will be to ensure that all content is properly sanitised prior to displaying it to the end user.

The page and label modules however differ in inferred intent from all other content creation sources within Totara are they are designed to facilitate standalone content creation. We understand that of those who do rely upon the ability to embed advanced content, many rely upon it just for labels and/or pages. As such each of these activities has its own setting Allow XSS that can be turned on at the site level, and which will allow content creators to embed any HTML they want into these two modules.

These settings are disabled by default, and can only be turned on by Site Administrators.

If the Disable consistent cleaning setting is turned on then these settings will not be shown as they are would have no effect.

Changes made by TL-20729.

Format string now encodes all potentially dangerous characters 

All titles and headings in Totara are passed through format_string. As of Totara 13 all content passed through format_string will be have potentially dangerous characters to ensure the content is safe to display to the end user.

There is no way to restore the previous behaviour. If you encounter problems due to characters being encoded by format_string then you need to review the uses of format_string leading to these problems. Only user content should be passed through format_string, and HTML tags are not permitted for user content intended for titles and headings.

Changes made by TL-20704.

Removal of the "Remove HTML tags from all activity names" setting

This setting was on by default. When turned on any < and > characters within activity names were encoded in order to ensure HTML tags would not be displayed within activity titles.

As per the above change (Format string now encodes all potentially dangerous characters) these characters are now consistently encoded to ensure that strings are safe to display to the end user. This setting therefore has no effect, as content passed through format_string is always and consistently encoded.

There is no alternative, or way to restore previous behaviour for this change.

Changes made by TL-20704.

User' full name is consistently encoded before display

A new function, clean_string, has been introduced that encodes all potentially dangerous characters so that the string is safe for display to the end user.

It is now used when before displaying a users full name within the interface in order to ensure that all potentially dangerous characters are encoded.

Changes made by TL-20704.

Filters can now mark themselves as compatible with HTML purifier

A new method is_compatible_with_clean_text() has been added to the moodle_text_filter base class. All filters in core have been updated to override this method, and all third party filters should be updated in the future. The method allows the filter to state whether its results are compatible with HTML purifier or not.

If all of the filters enabled on a site are compatible with HTML purifier, then all content passed through format_text() will be cleaned using HTML purifier before being returned.

If any of the filters enabled on the site are not compatible with HTML purifier then content passed through format_text will be cleaned prior to HTML purifier being run.

Changes made by TL-20729.

Consequences of these changes

Content is now consistently sanitised before being loaded into the editor unless the Disable consistent cleaning setting has been turned on. This has been done to protect the end user from cross-user XSS attacks through editable content.

However it comes with a risk of dataloss for any site that upgrades to Totara 13 and meets all of the following criteria:

• Has user created content that contains advanced HTML tags in any of the areas noted below as affected by calls to format_text()
• The Disable consistent cleaning setting was not turned on during upgrade

When a user edits something that contains advanced HTML the content will be sanitised, removing any content that is not permitted. When the user saves this area, even if they did not change the affected field, it will save the sanitised content back to the database. After which the original content that included the advanced HTML will be lost.

If you have a site that has used advanced HTML within your content then you will need to either turn the Disable consistent cleaning setting on, or update your content so that it no longer uses advanced HTML.

If you are dependent upon advanced HTML but do not wish to turn the Disable consistent cleaning setting on then our advice would be to look at implementing filters that would convert a predefined syntax into HTML for you. This could make it both easier to embed advanced content, and safer than turning on the setting.

Areas affected by this change

Calls to format_string()

All content passed through format_string() will now be encoded. This affects all titles, and headings in the product, as well as any other places relying upon this function to clean content.

Calls to format_text()

The following areas are affected by the changes to format_text. If advanced HTML tags are used in these areas they will be removed, unless Disable consistent cleaning has been turned on.

• Site summary
• Category description
• Program/Certification summary
• Course summary
• Course section description
• Group description
• Grouping description
• Module intro (description)
• Forum posts
• Quiz feedback
• Lesson questions and pages (not user answers)
• Database module templates
• HTML block
• Login instructions
• Scale description (grades)
• Outcome description (grades)
• Question category description
• Question text
• Question behaviour: comments