Diligence is still under development and incomplete, and some this documentation is wrong. For a more comprehensive but experimental version download the Savory Framework, which was the preview release of Diligence.
Backup Service
This service lets you do a live export of your MongoDB databases and collections to JSON, optionally gzipping the output to save space. You can set up your application's "crontab" to have the backup run regularly.
Backups are very fast: large databases can be fully exported in durations measured in seconds or minutes.
You might wonder what advantages this service has over MongoDB's mongodump or mongoexport tools. First, from our experience, the admin tools that come with MongoDB are overly simplistic and unreliable. Otherwise, Diligence's backup service offers the following advantages:
- Thoughput: Because we're using the Java MongoDB driver underneath, with its support for connection pooling, we can achieve much higher throughput than the command line tools, which use a single connection and no concurrency. The default is to use 5 threads (and thus 5 connections at most) at once.
- True JSON: The mongoexport tool does not export a real JSON array, instead it exports each document as a JSON dict, separating each document with a newline. Diligence exports a standard JSON array, readable from any standard JSON parser.
- Consistency: Works with the same MongoDB connection as your application, guaranteeing that you're backing up exactly the same data your application sees. This is especially important in a sharded or replica set deployment.
- Operations: You don't have to create system scripts to backup your DB. Instead, you can stay in JavaScript and Diligence. You do not even need MongoDB or its command line tools installed.
- Iterators: The backup service uses Iterators, so you can transform your data in various ways while backing up, or even include non-MongoDB data.
Usage
Make sure to check out the API documentation for Diligence.Backup.
To export the whole database:
document.executeOnce('/diligence/service/backup/')
Diligence.Backup.exportMongoDb({directory: '/tmp/diligence-backup/'})
The API further lets you select the MongoDB database and collections you wish to export, otherwise by default it uses the current default database and goes through all collections. You can also set "gzip" to true in order to gzip the resulting files.
To schedule the backup to run every day at 6am, add this to your "crontab":
6 <% document.executeOnce('/diligence/service/backup/'); Diligence.Backup.exportMongoDb({directory: '/tmp/diligence-backup/', gzip: true}); %>
To import a collection:
Diligence.Backup.importMongoDbCollection({file: '/tmp/diligence-backup/users.json'})
The collection name will be parsed from the filename. If the filename ends with ".gz", it will be assumed to be gzipped and unzipped accordingly. (For example "/tmp/diligence-backup/users.json.gz".) By default the imported documents will be merged into the collection: set the "drop" key true if you want the collection to be dropped before importing.
The Diligence Manual is provided for you under the terms of the Creative Commons Attribution-NonCommercial-ShareAlike 4.0 International License. The complete manual is available for download as a PDF.
