How to copy a collection from one database to another in MongoDB

Question

Is there a simple way to do this

User · Answer

There are different ways to do the collection copy. Note the copy can happen in the same database, different database, sharded database or mongod instances. Some of the tools can be efficient for large sized collection copying.

Aggregation with $merge: Writes the results of the aggregation pipeline to a specified collection. Note that the copy can happen across databases, even the sharded collections. Creates a new one or replaces an existing collection. New in version 4.2. Example: db.test.aggregate([ { $merge: { db: "newdb", coll: "newcoll" }} ])

Aggregation with $out: Writes the results of the aggregation pipeline to a specified collection. Note that the copy can happen within the same database only. Creates a new one or replaces an existing collection. Example: db.test.aggregate([ { $out: "newcoll" } ])

mongoexport and mongoimport: These are command-line tools. mongoexport produces a JSON or CSV export of collection data. The output from the export is used as the source for the destination collection using the mongoimport.

mongodump and mongorestore: These are command-line tools. mongodump utility is for creating a binary export of the contents of a database or a collection. The mongorestore program loads data from a binary database dump created by mongodump into the destination.

db.cloneCollection(): Copies a collection from a remote mongod instance to the current mongod instance. Deprecated since version 4.2.

db.collection.copyTo(): Copies all documents from collection into new a Collection (within the same database). Deprecated since version 3.0. Starting in version 4.2, MongoDB this command is not valid.

NOTE: Unless said the above commands run from mongo shell.

Reference: The MongoDB Manual.

You can also use a favorite programming language (e.g., Java) or environment (e.g., NodeJS) using appropriate driver software to write a program to perform the copy - this might involve using find and insert operations or another method. This find-insert can be performed from the mongo shell too.

You can also do the collection copy using GUI programs like MongoDB Compass.

User · Answer

You can use aggregation framework to resolve your issue  db oldCollection aggregate    out    newCollection       It shoul be noted  that indexes from oldCollection will not copied in newCollection

User · Answer

Actually  there is a command to move a collection from one database to another  It s just not called  quot move quot  or  quot copy quot   To copy a collection  you can clone it on the same database  then move the cloned collection  To clone   gt  use db1 switched to db db1   gt  db source collection find   forEach        function x             db collection copy insert x                To move   gt  use admin switched to db admin   gt  db runCommand                    renameCollection   db1 source collection             to                 db2 target collection                The other answers are better for copying the collection  but this is especially useful if you re looking to move it

User · Answer

If RAM is not an issue using insertMany is way faster than forEach loop    var db1   connect   lt ip 1 gt   lt port 1 gt   lt db name 1 gt    var db2   connect   lt ip 2 gt   lt port 2 gt   lt db name 2 gt     var  list   db1 getCollection  collection to copy from   find     db2 collection to copy to insertMany  list toArray

User · Answer

I know this question has been answered however I personally would not do  JasonMcCays answer due to the fact that cursors stream and this could cause an infinite cursor loop if the collection is still being used  Instead I would use a snapshot     http   www mongodb org display DOCS How to do Snapshotted Queries in the Mongo Database   bens answer is also a good one and works well for hot backups of collections not only that but mongorestore does not need to share the same mongod

User · Answer

I d usually do   use sourcedatabase  var docs db sourcetable find    use targetdatabase  docs forEach function doc    db targettable insert doc

User · Answer

use  Studio3T for MongoDB  that have Export and Import tools by click on database   collections or specific collection  download link   https   studio3t com download

User · Answer

I would abuse the connect function in mongo cli mongo doc  so that means you can start one or more connection  if you want to copy customer collection from test to test2 in same server  first you start mongo shell  use test var db2   connect  localhost 27017 test2     do a normal find and copy the first 20 record to test2   db customer find   limit 20  forEach function p    db2 customer insert p         or filter by some criteria  db customer find   active   1   forEach function p    db2 customer insert p         just change the localhost to IP or hostname to connect to remote server  I use this to copy test data to a test database for testing

User · Answer

This might be just a special case  but for a collection of 100k documents with two random string fields  length is 15-20 chars   using a dumb mapreduce is almost twice as fast as find-insert copyTo   db coll mapReduce function     emit this  id  this      function k vs    return vs 0        out    coll2

User · Answer

This can be done using Mongo s db copyDatabase method   db copyDatabase fromdb  todb  fromhost  username  password    Reference  http   docs mongodb org manual reference method db copyDatabase

User · Answer

In case some heroku users stumble here and like me want to copy some data from staging database to the production database or vice versa here s how you do it very conveniently  N B  I hope there s no typos in there  can t check it atm   I ll try confirm the validity of the code asap    to app  The name of the app you want to migrate data to  from app  The name of the app you want to migrate data from  collection  the collection you want to copy  mongohq url  heroku config get --app   to app  MONGOHQ URL  parts   echo  mongohq url   sed  s mongodb   heroku       sed  s        g    to token   parts 0    to url   parts 1    to db   parts 2   mongohq url  heroku config get --app   from app  MONGOHQ URL  parts   echo  mongohq url   sed  s mongodb   heroku       sed  s        g    from token   parts 0    from url   parts 1    from db   parts 2   mongodump -h   from url  -u heroku -d   from db  -p  from token  -c   collection  -o col dump mongorestore -h   prod url  -u heroku -d   to app  -p  to token  --dir col dump   col dump   collection  bson -c   collection

User · Answer

for huge size collections  you can use Bulk insert    var bulk   db getSiblingDB dbName  targetCollectionName  initializeUnorderedBulkOp    db getCollection sourceCollectionName  find   forEach function  d        bulk insert d       bulk execute      This will save a lot of time   In my case  I m copying collection with 1219 documents  iter vs Bulk  67 secs vs 3 secs

User · Answer

In my case  I had to use a subset of attributes from the old collection in my new collection  So I ended up choosing those attributes while calling insert on the new collection   db  lt sourceColl gt  find   forEach function doc         db  lt newColl gt  insert            new field1  doc field1           new field2  doc field2

User · Answer

At the moment there is no command in MongoDB that would do this  Please note the JIRA ticket with related feature request   You could do something like   db  lt collection name gt  find   forEach function d   db getSiblingDB   lt new database gt      lt collection name gt    insert d         Please note that with this  the two databases would need to share the same mongod for this to work    Besides this  you can do a mongodump of a collection from one database and then mongorestore the collection to the other database

User · Answer

If between two remote mongod instances  use     cloneCollection    lt collection gt    from    lt hostname gt    query     lt query gt     copyIndexes   lt true false gt       See http   docs mongodb org manual reference command cloneCollection

User · Answer

You can always use Robomongo  As of v0 8 3 there is a tool that can do this by right-clicking on the collection and selecting  Copy Collection to Database   For details  see http   blog robomongo org whats-new-in-robomongo-0-8-3   This feature was removed in 0 8 5 due to its buggy nature so you will have to use 0 8 3 or 0 8 4 if you want to try it out

User · Answer

Using pymongo  you need to have both databases on same mongod  I did the following     db   original database  db2   database to be copied to  cursor   db   lt collection to copy from gt    find   for data in cursor      db2   lt new collection gt    insert data

User · Answer

The best way is to do a mongodump then mongorestore  You can select the collection via  mongodump -d some database -c some collection   Optionally  zip the dump  zip some database zip some database   -r  and scp it elsewhere  Then restore it  mongorestore -d some other db -c some or other collection dump some collection bson  Existing data in some or other collection will be preserved  That way you can  quot append quot  a collection from one database to another  Prior to version 2 4 3  you will also need to add back your indexes after you copy over your data  Starting with 2 4 3  this process is automatic  and you can disable it with --noIndexRestore

User · Answer

This won t solve your problem but the mongodb shell has a copyTo method that copies a collection into another one in the same database   db mycoll copyTo  my other collection      It also translates from BSON to JSON  so mongodump mongorestore are the best way to go  as others have said

[mongodb] How to copy a collection from one database to another in MongoDB

Examples related to mongodb

Examples related to mongo-collection