Command execution API

Owned by Former user (Deleted)
Last updated: Aug 26, 2020

Summary

This classes within the 'core\command' namespace are intended to be used when Totara code needs to interact with external programs. Typically, a developer might use built-in PHP functions such as exec(), shell_exec() or a number of others which execute commands on the shell (command line). By having these classes, we can provide additional security on top of functions such as those. If future issues arise in this domain, we should only have to fix them in one place rather than everywhere that could trigger an external command.

Risks that we can protect against by centralising this code could include, among other things, harmful commands appearing on the command line due to differences between operating systems for characters used for escaping, preventing a problem program from being run by excluding it from the white list or being able to easily identify where we can restrict the types of arguments provided to a program.

Using the API in new code

To run a program in new code:

1. Create an instance of \core\command\executable.  The first argument in the constructor is the full path to the binary or executable file to be run.

2. Add arguments using the following methods:

  • add_argument() - this is for adding a key/value pair. e.g. to get " format 'landscape' ". The $key argument and $operator argument are NOT sanitised. The $value argument will be validated according to the $paramtype argument.
  • add_switch() - this is to add a switch, e.g. to add "-v" to get version information. In that example, the dash should be supplied (you cannot just add "v" as the code does not know to add the dash itself). This value is NOT sanitised.
  • add_value() - this is to add a value on it's own without a key. e.g. in "php path/to/file", you might say 'path/to/file' is a value supplied on its own. The value will be validated according to the $paramtype argument.

The arguments remain in the order they are supplied. If they are executed as a shell command, they would appear on the command line in the same order they were added.

3. Optionally set whether the STDERR stream should be redirected to STDOUT, so that it can be found in the output, using the method redirect_stderr_to_stdout(). This is equivalent to adding '2>&1' to the end of a shell command. Don't do this if it's not necessary.

4. Run the command using one of the following methods:

  • execute() - this is the preferred method if it can be used since this will use the PCNTL library if available (see below). Internally this uses the PHP function exec() and can also replace shell_exec, system() and commands within back tick operators.
  • passthru() - if the ability to get the raw output of the command is required, use this method. Internally, this uses the PHP function passthru(). This does not use the PCNTL library.
  • popen() - if the ability to interact with the program using a file pointer is required, use this method. Internally, this uses the PHP function popen(). This does not use the PCNTL library.
  • proc_open() - this is similar to popen, but allows more control. Internally, this uses the PHP function proc_open(). This does not use the PCNTL library.

5. Optionally get the return status after execution using the method get_return_status(). Remember that typically 0 indicates success. Other integers may indicate various other errors.

6. Optionally get the output after execution using the method get_output(). This will be an array with each line of output separated into different elements in the array. Only the execute() method fills this array.

7. Whitelist the executable.

Example:

$exe = new \core\command\executable('/path/to/exe');
$exe->add_argument('key1', 123, PARAM_INT);
$exe->add_switch('switch1');
$exe->execute();
if ($exe->get_return_status() === 0) {
  $output = $exe->get_output();
}


In the above code, the PARAM_INT supplied to add_argument will mean the $value is confirmed to be an integer. An exception of type \core\command\exception will be thrown if it's not.

On a Unix command line, the above code would execute a shell command that would look like so:  '/path/to/exe' switch1 key1 '123'

White listing

If the new code is going into core. The path to the program will need to be added to the get_whitelist() method inside the \core\command\executable class. In order for this to work across operating systems, this would be a setting, so see how other code gets this path. Likely from a $CFG setting or using get_config().

If this is a plugin, this can be added to $CFG→thirdpartyexeclist in the config.php file (this is described in config-dist.php).

In each of these cases, the path is provided as the key. The value of that array element will be true or false.

  • True indicates it can be executed when originating from a web request or being run via the command line
  • False prevents it from being run when executed from the web, but can still be run via the command line

If code is added that does not include the program in the white list at all, running it via the command line will also not work. This obviously only applies when this API is used and doesn't prevent an external program being run by the built-in functions such as exec().

Validation

Validation is performed on values only (either when added on their own via add_value() or added as the value in a key/value pair via add_argument()).

Validation is NOT performed on keys, switches or operators. Therefore those should always be hard-coded if possible and at least never originate from user input.

'Values' should ideally not originate from user input if they don't have to. In some cases choices may be applicable such as the user choosing the format 'landscape' or 'portrait', in which case those are hard-coded and the user essentially chooses options 1 or 2, otherwise an exception is triggered. In other cases where we have to accept user input, make sure the input is properly santised.

Do not entirely rely on the validation provided in this library, as while it may prevent unwanted characters on the command line, for example, it may not prevent arguments that lead the program to perform unwanted behaviour internally, e.g. we allow an argument to only have alphanumeric characters, meaning something like 'version' might pass validation, but perhaps this also gives a malicious user information to use in an attack.

Validation is done in the method \core\command\argument class::set_value(). It will validate according to the supplied $paramtype. In your code, this will have been supplied to the add_value or add_argument methods of the executable object. Validation can be of the following types:

  • Checking against built-in paramtypes such as PARAM_INT or PARAM_ALPHANUM. The supplied value will be run through clean_param() and then checked for whether anything changes.
  • Checking against a regex. The regex must check from the start to beginning of the string, meaning it should use ^ and $. Only the 'i' and 's' modifiers are allowed.
  • Checking the full file path if the $paramtype is \core\command\argument:: PARAM_FULLFILEPATH. This will check if the supplied value starts with the values for filepaths defined in $CFG, including $CFG→dataroot, localcachedir, tempdir, cachedir and dirroot. This means the path can start with characters we wouldn't typically account for, such as in C:\ for Windows. The remainder of the path must conform to rules for PARAM_PATH. However, the direction of slashes is ignored, e.g. we allow Windows backslash in any OS.

If validation is failed. An exception is thrown, typically this would be \core\command\exception, however a \moodle_exception may be thrown in some cases if it arose from clean_param().

Escaping

When adding values, an $escape_ifnopcntl argument is available. If the PCNTL library is being used, values will not need to be escaped, and so they won't be. Otherwise, on the command line, values will be escaped.

If you are sure that the values do not need to be escaped, i.e. it's not user input, and it is required that these values are not escaped, even on the command line, set this value to false to allow this.

PCNTL

The PCNTL library is typically only available when PHP is executed on the command line and on Unix. It is an extension that will need to be installed and enabled. The command execution API described here will allow it to be run even with requests sent to the web server rather than just via the CLI. However, it is not available on Windows.

Using the pcntl_exec() function from this library, the program is executed by creating a new process and passing arguments to it directly rather than via the command line. This reduces the risk of unwanted commands being executed via the shell.

If the PCNTL library is enabled for a version of PHP on the server, you allow the Command Execution API to run the executed program via this library, including when the code originates from a web request. To do so, find the full path to the PHP binary for which PCNTL is enabled. Add this pathname to $CFG→pcntl_phpclipath. When a program is executed via this API with valid arguments, a script will run the command by calling this version of PHP to run pcntl_exec().

While using PCNTL reduces risks associated with insufficient escaping, other risks such as how the program itself deals with supplied arguments still apply (see the 'version' example in the Validation section above).