Ways to implement data versioning in MongoDB

Question

Can you share your thoughts how would you implement data versioning in MongoDB   I ve asked similar question regarding Cassandra  If you have any thoughts which db is better for that please share   Suppose that I need to version records in an simple address book   Address book records are stored as flat json objects   I expect that the history    will be used infrequently will be used all at once to present it in a  time machine  fashion there won t be more versions than few hundred to a single record  history won t expire    I m considering the following approaches    Create a new object collection to store history of records or changes to the records  It would store one object per version with a reference to the address book entry  Such records would looks as follows         id    new id     user   user id    timestamp   timestamp    address book id    id of the address book record     old record     first name    Jon    last name   Doe           This approach can be modified to store an array of versions per document  But this seems to be slower approach without any advantages  Store versions as serialized  JSON  object attached to address book entries  I m not sure how to attach such objects to MongoDB documents  Perhaps as an array of strings   Modelled after Simple Document Versioning with CouchDB

User · Answer

I worked through this solution that accommodates a published, draft and historical versions of the data:

{
  published: {},
  draft: {},
  history: {
    "1" : {
      metadata: <value>,
      document: {}
    },
    ...
  }
}

I explain the model further here: http://software.danielwatrous.com/representing-revision-data-in-mongodb/

For those that may implement something like this in Java, here's an example:

http://software.danielwatrous.com/using-java-to-work-with-versioned-data/

Including all the code that you can fork, if you like

https://github.com/dwatrous/mongodb-revision-objects

User · Answer

There is a versioning scheme called  Vermongo  which addresses some aspects which haven t been dealt with in the other replies   One of these issues is concurrent updates  another one is deleting documents   Vermongo stores complete document copies in a shadow collection  For some use cases this might cause too much overhead  but I think it also simplifies many things   https   github com thiloplanz v7files wiki Vermongo

User · Answer

If you are using mongoose  I have found the following plugin to be a useful implementation of the JSON Patch format  mongoose-patch-history

User · Answer

Another option is to use mongoose-history plugin   let mongoose   require  mongoose    let mongooseHistory   require  mongoose-history    let Schema   mongoose Schema   let MySchema   Post   new Schema       title  String      status  Boolean      MySchema plugin mongooseHistory      The plugin will automatically create a new collection with the schema name     history      In this case  collection with name  my schema history  will be created

User · Answer

The first big question when diving in to this is  how do you want to store changesets     Diffs  Whole record copies    My personal approach would be to store diffs  Because the display of these diffs is really a special action  I would put the diffs in a different  history  collection   I would use the different collection to save memory space  You generally don t want a full history for a simple query  So by keeping the history out of the object you can also keep it out of the commonly accessed memory when that data is queried   To make my life easy  I would make a history document contain a dictionary of time-stamped diffs  Something like this          id    id of address book record       changes                      1234567      city     Omaha    state     Nebraska                     1234568      city     Kansas City    state     Missouri                         To make my life really easy  I would make this part of my DataObjects  EntityWrapper  whatever  that I use to access my data  Generally these objects have some form of history  so that you can easily override the save   method to make this change at the same time   UPDATE  2015-10  It looks like there is now a spec for handling JSON diffs  This seems like a more robust way to store the diffs   changes

User · Answer

If you re looking for a ready-to-roll solution -  Mongoid has built in simple versioning  http   mongoid org en mongoid docs extras html versioning  mongoid-history is a Ruby plugin that provides a significantly more complicated solution with auditing  undo and redo  https   github com aq1018 mongoid-history

User · Answer

Here s another solution using a single document for the current version and all old versions          id  ObjectId             data              vid  1  content   foo               vid  2  content   bar              data contains all versions  The data array is ordered  new versions will only get  pushed to the end of the array  data vid is the version id  which is an incrementing number   Get the most recent version   find          id  ObjectId                  data     slice -1         Get a specific version by vid   find          id  ObjectId                  data     elemMatch    vid  1           Return only specified fields   find          id  ObjectId                  data     elemMatch    vid  1       data content  1       Insert new version   and prevent concurrent insert update   update                  id  ObjectId                  and                  data vid     not    gt 2                       data vid  2                           push    data     vid  3   content   baz            2 is the vid of the current most recent version and 3 is the new version getting inserted  Because you need the most recent version s vid  it s easy to do get the next version s vid  nextVID   oldVID   1   The  and condition will ensure  that 2 is the latest vid   This way there s no need for a unique index  but the application logic has to take care of incrementing the vid on insert   Remove a specific version   update          id  ObjectId                  pull    data     vid  2           That s it    remember the 16MB per document limit

User · Answer

I have used the below package for a meteor MongoDB project  and it works well  the main advantage is that it stores history revisions within an array in the same document  hence no need for an additional publications or middleware to access change-history  It can support a limited number of previous versions  ex  last ten versions   it also supports change-concatenation  so all changes happened within a specific period will be covered by one revision    nicklozon meteor-collection-revisions  Another sound option is to use Meteor Vermongo  here

[mongodb] Ways to implement data versioning in MongoDB

Examples related to mongodb

Examples related to database-versioning