Pagination

Some endpoints support pagination, and to ensure stable, reliable, and efficient performance, we use ‘seek’ pagination. The main advantage of ‘seek’ pagination is that the items within a given page remain stable, even when new data is added, allowing for better performance optimization.

The after query parameter is used to retrieve all items that come after a specified item. In most cases, the id of an item is used for the after parameter, as shown in the example below. The documentation for each endpoint specifies which field should be used for the after parameter.

When polling our paginated endpoints periodically, use the last retrieved value for the next request. Since the after parameter is interpreted in a strict “greater than” (>) sense, an empty page might be returned, indicating there are no new items.

For consecutive pages, the link_next field in the response contains the relative URI to fetch the next set of results. Additionally, the Link: header provides the same relative URI for the next page.

Example of a current page result:

{
    "items": [...],
    "page_size": 500,
    "link_next": "/main/v1/msp-chargesession?after=NLTD1_608C9A13-E3DC-4353-8BEE-7437305BC8D7"
}

To retrieve the next page, use the provided link_next value for your next request: GET https://example-api.on-tandemdrive.com/main/v1/msp-chargesession?after=NLTD1_608C9A13-E3DC-4353-8BEE-7437305BC8D7

Be aware that providing an invalid after value, such as a non-existent identifier, will also result in a 400 Bad Request error.

Initial request

If you want to retrieve all items from the start, you can omit the after parameter in your initial request. Alternatively, if you only want to fetch items created after a specific date and time, use the created_gt query parameter. In this case, the response will not contain any items, but it will include a link_next to the first page of results after the given date-time. If no such items exist, the API will return a “400 Bad Request” error.

It’s crucial to understand that the created_gt parameter is mutually exclusive with all other parameters, including after. Apply any further filtering criteria by appending them as query parameters to the received link_next URL in subsequent requests.

Keep in mind that the created_gt parameter should only be used for the very first request; subsequent pagination and polling should be handled using the after parameter. This requires the API client to maintain a record of the last item id received and include it in the after parameter for the next iteration.

Page size

If one wishes to use a page_size different from the default (currently 500 for most endpoints), the page_size parameter can optionally be specified. See the specification for minimum, maximum and default values for this parameter.

Pagination how-to

Below you’ll find a set of basic pagination examples. All examples use the GET /customer endpoint, but the techniques apply to all endpoints that use pagination.

First endpoint use

To start pagination through a set of results provided by a particular endpoint, use the endpoint URL without any parameters.

$ curl -u $public_id:$secret "$base_url/main/v1/customer"

This results in a page of customer data:

{
  "items": [
    {
      "id": "cd1ddcf5-9858-4f90-b132-094e505c0725",
      "admin_id": "graant",
      "name": "Mr. Grain",
      "country": "NL",
      "entity_type": "person",
      "external_reference": "R00001"
    },
    ...
    {
      "id": "8bb33876-d098-429e-b9ed-77f81ce6db00",
      "admin_id": "silaye",
      "name": "Gardening Enterprise",
      "country": "GB",
      "entity_type": "organisation",
      "company_reg_num": "6984621"
    }
  ],
  "page_size": 500,
  "link_next": "/main/v1/customer?after=8bb33876-d098-429e-b9ed-77f81ce6db00"
}

Retrieve the next page using the link_next value if more data is available. If the number of items is less than page_size, no more data is available. It is important to store the id of the last item so that data retrieval can be continued without duplicates.

Retrieving data starting at a certain moment in time

The created_gt parameter can be used to retrieve data created after the provided moment in time. This is only meant to be used once, normal pagination must use the after parameter.

Note that since the created_gt parameter is used in a strict > sense, it may make sense to choose a moment which is a bit earlier. This helps with including all required records in subsequent requests.

$ curl -u $public_id:$secret "$base_url/main/v1/customer?created_gt=2021-12-20T00%3A00%3A00Z"

{
  "items": [],
  "link_next": "/main/v1/customer?after=cd1ddcf5-9858-4f90-b132-094e505c0725",
  "page_size": 500
}

The first page of customers created after a given moment can be retrieved using:

$ curl -u $public_id:$secret "$base_url/main/v1/customer?after=cd1ddcf5-9858-4f90-b132-094e505c0725"

The created_at parameter of the customer can be used to filter any customers that were created between 20221-12-20 and the 2022-01-01. From this point onward the scenario “First endpoint use” can be used.

Retrieving new data

This scenario implies one has already obtained data using either of the scenarios above. Use the id from your local data to fetch the last customer retrieved. We represent the id as 9999. New data can be retrieved using 9999 as the value for the after parameter:

$ curl -u $public_id:$secret "$base_url/main/v1/customer?after=9999"

Keyboard shortcuts