Deadline Database Backup Strategy
Version: Deadline 7.0 and later
INTRODUCTION
The Deadline Database is a key component of your Deadline render farm, and for that reason, it's recommended to back up the database regularly. This will ensure you always have a backup in case the database becomes corrupted due to a power surge or hardware failure. In addition, this process can be useful if you ever need to take a snapshot of the database.
The idea of backing up the database is often overlooked, but today we are here to provide a quick and easy tutorial on how to set up one-time and automatic Deadline database backups! For the purposes of this tutorial, we will be focusing on Windows and Linux operating systems.
BACKING UP THE DEADLINE DATABASE
To create a single backup of the database, open up a Terminal (Linux) or Command Prompt (Windows), and then change directories to the Deadline database bin folder (ie: C:\DeadlineDatabase9\mongo\application\bin). Once you are in the bin directory, you can execute this command:
mongodump -d DATABASE_NAME -o PATH_TO_BACKUP_FOLDER
The DATABASE_NAME parameter is the name of the database you want to back up. By default, the Deadline database name is deadlineNdb, where N is the Deadline version number (ie: deadline8db or deadline9db). In addition, the PATH_TO_BACKUP_FOLDER parameter is the folder you want to save the backup to. For example, if you wanted to back up your Deadline 9 database and save it to C:\Backups, you would run this command:
mongodump -d deadline9db -o C:\Backups
While this will do a one-time backup of the Deadline database, wouldn't it be nice to create automatic backups on a daily basis? In the next two sub-sections we will cover how to do this in both Windows and Linux environments using the following Python script. Note that this assumes you have Python installed on the machine.
from subprocess import call
import datetime
import os
curr_date = datetime.datetime.today().strftime('%Y-%m-%d_%Hh-%Mm-%Ss')
call(["mongodump", "-d", "deadline9db", "-o", os.path.join( "C:\\Backups", curr_date )])In this script, we are continuing our example of backing up the deadline9db database. However, we are now backing up to a sub-folder in C:\Backups with the current timestamp. If this script were to run at noon on April 20, 2017, it will create a folder called C:\Backups\2017-04-20_12h-00m-00s which containing BSON and JSON backups of your database.
Note that this script is assuming that it's being run from the Deadline database bin folder, so you should create it in that location before continuing with this tutorial, and name it backup_database.py. Or, as an alternative, you can specify the full path to mongodump in the call command:
call(["C:\\DeadlineDatabase9\\mongo\\application\\bin\\mongodump", "-d", "deadline9db", "-o", os.path.join( "C:\\Backups", curr_date )])
If you have multiple databaes and you want to back up all of them, you can remove the "-d" and "deadline9db" parameters from the call command above:
call(["mongodump", "-o", os.path.join( "C:\\Backups", curr_date )])
WINDOWS
To create automatic backups in Windows, we will take advantage of the Windows Task Scheduler. To begin, open up the Task Scheduler, which can be done by clicking on the Windows Start menu and searching for "Task Scheduler". When the Task Scheduler window opens, click on Task Scheduler (Local) on the left hand side, and then click Create Task... on the right hand side. This will bring up the Create Task dialog.
Under the General tab, we will want to give our task an appropriate name such as Deadline Database Backup.
Next, select the Triggers tab and press the New... button. Here we will want to set the time for when our automatic task will run. In this example, I selected Daily at 12:00:00pm. Click on OK to add the trigger.
Now select the Action tab and press the New... button. Here we will want to configure the action to execute our backup script. In the Action drop down, choose Start a program. For the Program/script, browse to the location of the Python executable on the machine. For the Arguments, specify the full path to the backup_database.py script file. Click on OK to add the action.
Finally, click the OK button on the Create Task dialog to save you task, and you're done!
LINUX
To create daily backups in a Linux environment, we will take advantage of the cron daemon. Begin by opening up a Terminal and executing the command:
crontab -e
At this point you may be prompted to choose a text editor, so select the one which you prefer. We will be adding in a new cron entry at the bottom of the file. Like the Windows example above, we'll be running this script every day at noon, so we'll add this line to the file:
0 12 * * * /usr/bin/python /opt/Thinkbox/Deadline9Database/mongo/application/bin/backup_database.py
After saving the file, you're done!
RESTORING THE DATABASE
If your Deadline database ever becomes corrupted, or you need to restore to a previous state, it's as simple as executing the following command in a Terminal or Command Prompt:
mongorestore PATH_TO_THE_BACKUP
For example, to restore the backup that was created at noon on April 20, 2017, we would run this command:
mongorestore C:\Backups\2017-04-20_12h-00m-00s
One thing to keep in mind though is that doing a restore will revert ALL data in the database back to this previous state. If you want to keep the current information for debug purposes, do a backup before you restore!
WRAPPING UP
Now that you have finished this tutorial, creating backups of your database should be pretty straightforward. One thing to keep in mind though is the size of the backups. After this script has run for a few months, storage space might start to become an issue. For those who are comfortable coding in Python, I've left this as a fun exercise for you to modify the script above to remove old backups that are older than 3 months.
For additional readings please refer to: