MongoDB point-in-time backup and recovery.

In this blog, we describe a procedure that allows you to manually perform a point-in-time restore of MongoDB data. Point-in-time restore refers to the ability to restore the database to a precise moment in time. The instructions presented here require that you have regular backups of your data, and that your data is stored in a replica set.

Requirements :mongodb22

  1. You need a backup of the database – this can be either from a mongodump or from a file system copy.
  2. You need access to an oplog that contains data that goes back to when the backup was made. For example if you made a backup 24 hours ago, then you need an oplog that goes back at least 24 hours. This oplog will be extracted from one of the replica set members in the live database.

Mongodump is the base logical backup tool included with MongoDB. It takes a full BSON copy of database/collections, and optionally includes a log of changes during the backup used to make it consistent to a point in time. Mongorestore is the tool used to restore logical backups created by Mongodump. I’ll use these tools in the steps in this article to restore backed-up data. This article assumes a mongodump-based backup that was taken consistently with oplog changes (by using the command flag “–oplog”), and the backup is being restored to a MongoDB instance.

Example scenario :

We have a backup of mongodb backup of databases running in regular intervals.


Assuming you have oplog on your live/production database that goes back at least 24 hours (ie. an oplog window of at least 24 hours). The oplog is available on each node in the replica set, and can be dumped out from any one of the replica set members. The oplog contains a record of all database events that have recently occurred.

For testing we are added data to the collection and verified the current document count

rs0:PRIMARY> use testdata
switched to db testdata
rs0:PRIMARY> db.birthday.find().count()
rs0:PRIMARY> db.birthday.find().limit(1)
{ "_id" : ObjectId("5bf18634df205e1d523542fa"), "name" : "name1", "birthday" : ISODate("1974-09-18T00:00:00Z") }
rs0:PRIMARY> db.birthday.insertOne({name: "name99", birthday: ISODate("1990-02-11T02:00:00Z") }
... )
        "acknowledged" : true,
        "insertedId" : ObjectId("5bf19caeeded9b5c7021d2f3")
rs0:PRIMARY> db.birthday.find().limit(1)
{ "_id" : ObjectId("5bf18634df205e1d523542fa"), "name" : "name1", "birthday" : ISODate("1974-09-18T00:00:00Z") }
rs0:PRIMARY> db.birthday.find().count()


Now we are dropping the collection birthday


Restoration instructions:

  1. Verify the last time stamp in oplog
rs0:PRIMARY> use local
switched to db local
rs0:PRIMARY>{}, {ts: 1,}).sort({ts: -1}).limit(1)
{ "ts" : Timestamp(1542563322, 1) }

2. On one of the members of your live replica set, dump the oplog collection using mongodump:

[root@ip-172-31-93-199 mongodump]# mongodump -u admin -p XXXXXX --authenticationDatabase admin -d local -c -o oplogdump
2018-11-18T17:55:20.593+0000 writing to
2018-11-18T17:55:20.614+0000 done dumping (323 documents)
[root@ip-172-31-93-199 mongodump]# cd oplogdump/
[root@ip-172-31-93-199 oplogdump]# ls
[root@ip-172-31-93-199 oplogdump]# cd local/
[root@ip-172-31-93-199 local]# ls

It is not necessary Oplog will have a record created exact on the same time, it might create a record earlier or later to this time. And obviously that will happen most of the time. In that scenario, you must find an epoch value just less than the required epoch time and you will restore up to that timestamp. Your goal is to find “”ts”:{“timestamp”:{“t”:x,”i”:y}}” entry and note its values from Oplogs. X is the epoch value we are interested in and Y is an incrementing ordinal for operations within a given second. In case you find more than 1 value of Y for the same value of X, you should use the largest Y value.

3. Connect to the server using the mongo shell, find the bad operation in the oplog and make a note of the time_t:ordinal (timestamp) values. Alternatively, use bsondump to manually inspect the BSON dump of the oplog.

[root@ip-172-31-93-199 local]# bsondump > oplogrecovery.json
2018-11-18T18:05:49.359+0000    323 objects found
[root@ip-172-31-93-199 local]# ls

4.  Find the timestamp of the bogus transaction, which represents the point until you want to replay the oplog:

[root@ip-172-31-93-199 local]# tail -n300 oplogrecovery.json
{"ts":{"$timestamp":{"t":1542562302,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"6091652354277864417"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:31:42.796Z"},"o":{"msg":"periodic noop"}}
{"ts":{"$timestamp":{"t":1542562312,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"7661599134073288470"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:31:52.797Z"},"o":{"msg":"periodic noop"}}
{"ts":{"$timestamp":{"t":1542562322,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"-2999680384224794684"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:32:02.797Z"},"o":{"msg":"periodic noop"}}
{"ts":{"$timestamp":{"t":1542562342,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"8689417822344734569"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:32:22.797Z"},"o":{"msg":"periodic noop"}}
{"ts":{"$timestamp":{"t":1542562352,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"7595615792183348688"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:32:32.798Z"},"o":{"msg":"periodic noop"}}
{"ts":{"$timestamp":{"t":1542562362,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"1667414361006145572"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:32:42.798Z"},"o":{"msg":"periodic noop"}}
{"ts":{"$timestamp":{"t":1542562372,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"-4034500875267832055"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:32:52.798Z"},"o":{"msg":"periodic noop"}}
{"ts":{"$timestamp":{"t":1542562382,"i":1}},"t":{"$numberLong":"1"},"h":{"$numberLong":"7657953432518951006"},"v":2,"op":"n","ns":"","wall":{"$date":"2018-11-18T17:33:02.799Z"},"o":{"msg":"periodic noop"}}


In this case drop happened at time stamp {“t”:1542562328,”i”:1}  and we need to restore till {“t”:1542562322,”i”:1}

5. Start a new standalone mongod instance to rebuild the server and verify the data before applying it to production. If you use mongorestore to restore a backup.Start a new mongod process, and restore the last known good backup using mongorestore :

[root@ip-172-31-93-199 mongodump]# mongorestore -uadmin -p XXXXX --authenticationDatabase admin  -d testdata /mongodump/testdata/birthday.bson
2018-11-18T18:19:33.339+0000    checking for collection data in /mongodump/testdata/birthday.bson
2018-11-18T18:19:33.360+0000    reading metadata for testdata.birthday from /mongodump/testdata/birthday.metadata.json
2018-11-18T18:19:33.390+0000    restoring testdata.birthday from /mongodump/testdata/birthday.bson
2018-11-18T18:19:33.469+0000    no indexes to restore
2018-11-18T18:19:33.469+0000    finished restoring testdata.birthday (2000 documents)
2018-11-18T18:19:33.469+0000    done

Verify the data after dump restoration

s0:PRIMARY> use testdata
switched to db testdata
rs0:PRIMARY> show collections
rs0:PRIMARY> db.birthday.find().count()

mongorestore has restore 2000 documents out of 2001 data count

6.  Now you’re ready to replay the oplog using — oplogLimit to set the delimiter:

[root@ip-172-31-93-199 local]# mongorestore -u admin -p XXXXX --authenticationDatabase admin  --oplogReplay --oplogLimit  1542562322:1 /mongodump/oplogdump/local/
2018-11-18T18:28:29.182+0000    checking for collection data in /mongodump/oplogdump/local/
2018-11-18T18:28:29.182+0000    replaying oplog
2018-11-18T18:28:29.207+0000    done

7. Check that all documents are now back in the collection and data has been correctly restored.

rs0:PRIMARY> use testdata
switched to db testdata
rs0:PRIMARY> db.birthday.find().count()


8. Depending on the procedure used in the previous steps, restore the dump or the data files into the live/production replica set.



One thought on “MongoDB point-in-time backup and recovery.

Add yours

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s

Powered by

Up ↑

%d bloggers like this: