[rest] API pagination best practices

I've thought long and hard about this and finally ended up with the solution I'll describe below. It's a pretty big step up in complexity but if you do make this step, you'll end up with what you are really after, which is deterministic results for future requests.

Your example of an item being deleted is only the tip of the iceberg. What if you are filtering by color=blue but someone changes item colors in between requests? Fetching all items in a paged manner reliably is impossible... unless... we implement revision history.

I've implemented it and it's actually less difficult than I expected. Here's what I did:

  • I created a single table changelogs with an auto-increment ID column
  • My entities have an id field, but this is not the primary key
  • The entities have a changeId field which is both the primary key as well as a foreign key to changelogs.
  • Whenever a user creates, updates or deletes a record, the system inserts a new record in changelogs, grabs the id and assigns it to a new version of the entity, which it then inserts in the DB
  • My queries select the maximum changeId (grouped by id) and self-join that to get the most recent versions of all records.
  • Filters are applied to the most recent records
  • A state field keeps track of whether an item is deleted
  • The max changeId is returned to the client and added as a query parameter in subsequent requests
  • Because only new changes are created, every single changeId represents a unique snapshot of the underlying data at the moment the change was created.
  • This means that you can cache the results of requests that have the parameter changeId in them forever. The results will never expire because they will never change.
  • This also opens up exciting feature such as rollback / revert, synching client cache etc. Any features that benefit from change history.

Examples related to rest

Access blocked by CORS policy: Response to preflight request doesn't pass access control check Returning data from Axios API Access Control Origin Header error using Axios in React Web throwing error in Chrome JSON parse error: Can not construct instance of java.time.LocalDate: no String-argument constructor/factory method to deserialize from String value How to send json data in POST request using C# How to enable CORS in ASP.net Core WebAPI RestClientException: Could not extract response. no suitable HttpMessageConverter found REST API - Use the "Accept: application/json" HTTP Header 'Field required a bean of type that could not be found.' error spring restful API using mongodb MultipartException: Current request is not a multipart request

Examples related to pagination

How to use paginator from material angular? how to implement Pagination in reactJs In Flask, What is request.args and how is it used? Custom pagination view in Laravel 5 Simple pagination in javascript UITableView load more when scrolling to bottom like Facebook application How to use pagination on HTML tables? how to remove pagination in datatable Laravel Pagination links not including other GET parameters API pagination best practices

Examples related to api-design

When do I use path params vs. query params in a RESTful API? What are best practices for REST nested resources? API pagination best practices What is the proper REST response code for a valid request but an empty data?