I've thought long and hard about this and finally ended up with the solution I'll describe below. It's a pretty big step up in complexity but if you do make this step, you'll end up with what you are really after, which is deterministic results for future requests.
Your example of an item being deleted is only the tip of the iceberg. What if you are filtering by color=blue
but someone changes item colors in between requests? Fetching all items in a paged manner reliably is impossible... unless... we implement revision history.
I've implemented it and it's actually less difficult than I expected. Here's what I did:
changelogs
with an auto-increment ID columnid
field, but this is not the primary keychangeId
field which is both the primary key as well as a foreign key to changelogs.changelogs
, grabs the id and assigns it to a new version of the entity, which it then inserts in the DBchangeId
represents a unique snapshot of the underlying data at the moment the change was created. changeId
in them forever. The results will never expire because they will never change.