GraphQL schema file changes in Totara 17

Pre-Totara 17 GraphQL file structure

In Totara, GraphQL files are stored in the webapi folder within the specific component or plugin that is defining or using the service.

There are two types of files specific to GraphQL:

*.graphqls
GraphQL schema files, written in SDL, which provide the schema definition that specifies what is possible with the API.

*.graphql
Persistent GraphQL query files that define a specific query or mutation that can be referenced and used.

In Totara prior to version 17, all schema files were located directly within the webapi/ folder for the component. The files could have any filename - all .graphqls files would be collated from across components and plugins to build the final schema.

Some examples of valid schema paths:

./server/totara/api/webapi/schema.graphqls       # a specific component, providing all schema in one file
./server/mod/perform/webapi/activity.graphqls    # another component might break up the schema by task to make a large schema easy to manage
./server/lib/webapi/schema.graphqls              # the main core schema is located in the ./server/lib/webapi/ folder
./server/lib/webapi/course.graphqls              # core components are required to locate their schema in the ./server/lib/webapi folder (not within the component)

In Totara 16 and below, a single schema is generated for the site by combining all schema files from all components into a single master schema. This one schema is used by all endpoints (ajax, dev and mobile).

In addition to the schema definitions, components and plugins can define .graphql persistent queries. However, unlike the schema files, these persistent queries are specific to the schema endpoint they are designed to be used by. This is specified by locating these files in a subdirectory named after the relevant endpoint.

Some example persistent query files:

./server/totara/api/webapi/ajax/delete_client.graphql       # a persistent mutation for deleting an API client
./server/totara/mobile/webapi/mobile/me.graphql             # a query for the mobile endpoint to obtain information about the current user
./server/course/webapi/ajax/course.graphql                  # a query to return a course, defined by the course core component

Version 17 GraphQL file structure

In Totara 17, we opted to make it possible to specify schema files that were limited to a specific endpoint type (mobile, dev, ajax, external). The reason for this was that a lot of the schema defined was already specific to one use case. For example, there are parts of the schema defined specifically for use by the mobile app, and others that only make sense when used by the front end via the AJAX endpoint. Splitting the schema allowed the external API to specify a clean, organised schema for external consumption, independent of other API use cases.

To achieve this, the behaviour was modified so that schema files are now also detected and included when defined within the same subfolder that is used by persistent queries. This was done without changing the existing behaviour, e.g. schema files directly within the webapi/ folder will continue to be included in all schema types.

Some examples of possible schema paths with the new structure:

./server/totara/api/webapi/ajax/schema.graphqls       # This schema is only included in the schema for the ajax endpoint
./server/mod/perform/webapi/core.graphqls             # This schema would be included in all endpoint types
./server/totara/mobile/webapi/mobile/schema.graphqls  # A schema dedicated to the mobile endpoint only

A component can continue to split schemas into multiple files within any subdirectory (all files with the matching extension will be included) and can provide a mixture of both core and endpoint-specific files.

Issues caused by multiple schemas

There are a couple of issues to be aware of that are caused by having multiple distinct schemas.

Dependencies

Because not all schema files are included in a single master schema, it's possible to generate an endpoint schema that is missing dependencies. For example, you use a type in a query for the 'mobile' endpoint but the type was defined in the 'ajax' schema, so it's not available. The schema validation will be able to pick up any issues like this and should give a sensible error message explaining the problem.

To avoid this, you typically want to keep as much of your schema together as possible, and move things into core when necessary.

Duplication

Because there are multiple independent schema it is also now possible to define something twice in different schemas, and not immediately hit an error. However, the 'dev' schema endpoint is defined in such a way that it automatically includes the schema files from all schema types, so it will give an error if you reuse a name. So again, schema validation of all endpoint types should identify any issues.

We have a unit test which validates all available schema types, so any errors will be picked up by the build.

Initial reorganisation of schema files

At the time that the schema was split, all schema files were created to support two specific endpoint types: 'mobile' and 'ajax'. The majority of the schema needed for the mobile app was located in the server/totara/mobile/webapi/ folder, so that was relocated to be unique to the mobile endpoint.

Most of the remaining schema were implemented purely for use by our JavaScript front end, and in some cases were not suitable for external use. For example, they expect very specific argument values that are not easily known by an external user. Others were very bespoke queries specifically designed to return the data to generate a specific front-end experience. Queries and mutations like that didn't belong in the external API, or at least require review and testing before they should be included. So for now these schema elements were moved to be part of just the 'ajax' endpoint.

The remaining schema elements were core items such as core types, enums, scalars and other reusable items that should be available across schemas. These items were left in the core webapi/ directory. In some cases this required splitting schema files into two parts.

Recommendations for organising schema files

Here are some guidelines for how to organise your schema files:

• If everything in the schema is designed to only be used by a specific endpoint, just put the whole schema into the endpoint directory. It's probably best to start more specific if you're not sure, and move it to core later, rather than the other way around.
• If there are core elements such as object types, enums and scalars that you need to use from multiple endpoints, put them in core.
• Keep any specific queries and mutations (and the input types and result types associated with them) in the endpoint directory.
• If you get schema validation errors due to dependencies, move only what you need into core.
• If you've got queries and mutations that you feel are suitable and useful for external use, and have tested them, you could put them in core. Please ensure the schema has been well documented and complies with our API design guidelines.

Also be aware that due to the reorganisation you might need to provide a different patch for t17-release, or may hit merge conflicts when cherry picking between branches.