Task progress logging

In Totara 20 scheduled and ad-hoc tasks log meta data into the database. Every task execution will record the following information:

task component and class name
scheduled: 1/true for scheduled, 0/false for ad-hoc
start time and end time
success: 1/true for the task completing with no exception raised into the scheduler
error message: when the task failed, the details of the exception raised.
DB queries: the number of database queries that were executed during this task’s execution

Additionally, tasks may record information on their own execution:

items to process: the total number of items that there is to process
items processed: the total number of items that were successfully processed
items failed: the total number of items that failed to process
message: any message the task may log

This information can be useful to track progress on long running tasks, or capture trends on how many items each execution of the task processes. Over time more tasks will be amended to record these counts.

As the information in this log table may grow really quickly, the task cron_log_cleanup_task runs daily to remove successful (success = 1) task runs older than XXX days. Tasks that fail are not removed by the cleanup task.

A report source CRON history is available to report on this data.

Add item progress logging into a task

in the tasks’ execute function, create a new task_db_log_trace instance with the tasks’ log object (the record in the database)

/** @var task_log $log */
$log = $this->get_current_log();
if (!empty($log)) {
    $trace = new task_db_log_trace($log);
}

Establish how many items there are to progress, and record the start of processing: $trace->progress(0, 0, $total_to_process);

To log progress, simply call with the current counts: $trace->progress($completed_count, $failed_count); . You can pass null as any argument to leave the current value as is. For instance, $trace->progress(null, $failed_count);will not impact the completed counter.

Note that this will not always actually save into the database. The record is updated only if the last update in the database was more than 10 seconds ago, or if the $total_count is passed. This is to prevent rewriting the record too often.

Ensure the task execution is called with the final counts $trace->progress($completed_count, $failed_count, $total_count); to ensure the data is saved.

For example:

function do_work() {
  $records = $DB->get_records('table', ['needs_processing' => 1]);
  // Save the initial counts
  $trace->progress(0, null, count($records));
  $completed = 0;
  $failed = 0;
  foreach ($records as $record) {
      if ($record->process_record()) {
        $completed++;
      } else {
        $failed++;
      }
      // Save the current progress (may not actually write in the database every time)
      $trace->progress($completed, $failed);
  }
  // Save the final counts
  $trace->progress($completed, $failed, count($records));
}