Secrets Management in Deadline

  • Blog

Author: Chris Behan, Software Development Engineer, AWS Thinkbox

Introduction

AWS Thinkbox Deadline 10.1.10 introduced the new Secrets Management feature to Deadline, which allows you to securely store secrets that Deadline uses (ex. Passwords and API Keys), and control which Deadline Workers can access those secrets. With the Deadline Secrets Management feature, all data to be kept secret is encrypted and stored in a new secure storage layer of the database. This new secure storage layer has machine and user specific access controls that you can configure. The Secrets Management feature requires use of the Deadline Remote Connection Server (RCS) and does not support Direct Connection.

Why would you want to use the Deadline Secrets Management feature?

Here are a few example use cases: 

You want to....

  • Only allow specific render nodes access to your Deadline Usage-Based Licensing URL and Activation code for rendering jobs.
  • Restrict artist machines from accessing any “Secrets”
  • Only allow the RCS machine to access AWS Credentials used for the Spot Event Plugin
  • Only allow specific machines to retrieve a custom secret that is used by a custom plugin

Tutorial

The Secrets Management feature utilizes a new secret storage layer in the database to store and retrieve secrets. The creation of the new Secret Storage layer is done through the Deadline Repository installer (>= V10.1.10). This means that the Secrets Management feature can only be enabled during the installation, not afterwards. 

In this tutorial you will:

  • Install the Deadline Repository with Secrets Management Enabled
  • Install the Deadline Client and configure it to use Secrets Management
  • Register a machine and grant it a Client access role

Requirements

Deadline Repository Installer version >= 10.1.10

Deadline Client Installer version >= 10.1.10

Deadline Repository

Unless otherwise stated below, follow the steps outlined in the Database and Repository Installation guide.

Existing Installations

If you already have a MongoDB database that is configured to use TLS, follow the below steps, if not, see the New Installations section. Note: Secrets Management requires that your MongoDB database be TLS enabled.

1. Select the option to connect to an existing MongoDB database.

2. Choose to enable Secrets Management

3. Continue to Secrets Management Setup

New Installations

1. You will need to install a new MongoDB database on your repository machine.

2. After the MongoDB download and installation has been completed, you are given the opportunity to configure authentication for your Mongo Database. Use the default options of “Require client authentication via SSL/TLS” and “Use Client certificate for DB user authentication”. In addition we recommend entering a passphrase for the MongoDB client certificate.

Secrets Management Setup

3. Select the default option to “Enable Secrets Management (Strongly Recommended)”.

4. You will then be prompted to enter an Admin Username and password. These Admin credentials will be used to create the initial admin user, and will be required for modifying all the new identity and secret access controls added to Deadline. It is essential that you remember the credentials entered in this step. After you have entered the credentials and stored them in a secure location that you will remember, such as AWS Secrets Manager, click “Next”.

5. Your Repository installation with Secrets Management enabled is now ready to commence. Click “Next”. The Repository should install successfully.

Congratulations! You have successfully installed a Deadline Repository with Secrets Management. You can now rest assured knowing that all of your sensitive data in Deadline, such as passwords and API keys, is encrypted and has explicit machine level access control. Next up we will install and configure a Deadline Client to act as the Server and entry point to our newly secured Deadline Repository.

For additional information on installing the Deadline Repository, see our Database and Repository Installation Guide.

Deadline Client

Unless otherwise stated in the steps below, follow the Client Installation guide, selecting the recommended options. 

By the end of this section, you will have installed and configured a server machine. The server machine will act as the entry point to your render farm, ensuring all connecting clients are authenticated and communicating over encrypted channels. In addition, all Deadline clients that connect to the server machine will have a unique identity generated for them. This identity can then be used to control the level of access a machine has to “Secrets” (passwords and API keys) stored in Deadline.

1. Select the “Remote Connection Server” component, and click “Next”. This will install the Remote Connection Server, Deadline’s server component, which allows seamless and secure communication between the different components (Monitor, Worker, and License Forwarder) of Deadline. 

2. Next Select “Direct Connection”. The server machine, which will act as the entry point to your repository, needs a direct connection to the database and repository. All additional Deadline clients will communicate indirectly with the Database and repository through this server machine.

3. Use the default Database TLS Certificate path (this will be the same as the default certificate generation path in the Repository Installer) and enter the TLS Certificate Password that you entered during step 2 of the repository installation. 

4. You will be asked if you would like to configure this machine as the initial server machine, by granting it a server role and access to the master key that was generated during the Repository installation. Select the default option of “Assign server role and grant master key access.” (strongly recommended).

Deadline’s new Secrets Management feature introduces a new identity management system. Central to this system is the “master key”, a cryptographic key that will be used to encrypt all passwords and API keys in Deadline. Each identity will have varying levels of access to the master key, which is needed to decrypt the aforementioned passwords and API keys. 

In this step we are indicating to the installer that we would like to assign a server role to this machine along with granting it access to the master key. 

5. Enter the administrator credentials from step 3 of the repository installation.

6. Use the default options for configuring the Remote Connection Server, ensuring the “Require external Clients to use TLS checkbox is selected”. Deadline Secrets Management requires all communications to be encrypted over TLS 1.2, this ensures that no adversaries are able to intercept secrets in transit.

7. We recommend entering a passphrase for the newly generated Remote Connection Server Certificates. This password will be required, in addition to the generated certificate, for all clients connecting to the Remote Connection Server. We advise saving this password to a secure location, such as AWS Secrets Manager.

8. You are now ready to install the Deadline client. The installation should complete successfully. For additional information on the Deadline Client installation, see our Client Installation Guide.

Launching the Remote Connection Server

The Remote Connection Server acts as the entry point for all deadline clients (both workers and monitors) to access the repository and database. In this tutorial, you have configured your Remote Connection Server to require all connecting clients to present a Client TLS Certificate in order to connect to the Remote Connection Server. In addition, Deadline Secrets Management allows you to provide specific access control to client machines on your render farm. 

To start the Deadline Remote Connection Server, navigate to your deadline bin folder and double click “deadlinercs.exe”. On Linux, navigate to opt/Thinkbox/Deadline10/bin and run ./deadlinercs . Note that the user running the above commands must have access to the Deadline10client.pfx (\DeadlineDatabase10\certs).

You should see a terminal window containing the Remote Connection Server logs. The Remote Connection Server is now running. Ensure you do not close this terminal.

Connecting to the Remote Connection Server

Next we will connect a Deadline Monitor to the Remote Connection Server that we just started. From the Deadline Monitor you can submit jobs, configure your repository, and configure the access control of other machines in your Render Farm.

Navigate to your Deadline bin folder and double click “deadlinemonitor.exe”. On linux, navigate to /opt/Thinkbox/Deadline10/bin and run./deadlinemonitor. Note that the user running the above commands must have access to the Deadline10RemoteClient.pfx (\Thinkbox\Deadline10\certs\Deadline10RemoteClient.pfx).

After the Deadline Monitor launches, you will need to change your repository connection type. Click File → Change Repository.

Select the Remote Connection Server radio Button, enter 127.0.0.1 and port 4433, since your RCS is running locally and the default port for TLS is 4433. Check the “Use TLS/SSL” Checkbox, then select the client certificate that you generated during step 6 of the client installation section, along with the passphrase you set for the client certificate during step 6. Click OK.

Congratulations! You have now securely connected to the Deadline RCS. To connect other machines to the remote connection server you would follow the same steps as above, except instead of entering 127.0.0.1 as the ip address, you would enter the ip address of the machine as it appears to other machines on your network.

Managing Identities

Secrets in Deadline include:

  • Usage Based Licensing URL
  • Usage Based Licensing Activation Code
  • Mapped Drive Passwords
  • Email Notification SMTP credentials
  • Render As User Credentials
  • Event, Application, Cloud, or Balancer plugin parameters with the Control Type of “Password”

To view and manage identities, select Tools → Manage Identities... 

You will be prompted to enter the Administrator Credentials.

In this example, I have 3 external machines connected to my Remote Connection Server. Suppose I want to render a 3ds Max scene with Deadline UBL using IP-0A804EAD (a windows render node, highlighted above). I will need to register that machine’s identity, and give it a client role, since it will need access to the UBL credentials in the database, which are considered Secrets. 

1. Select the machine you will be registering from the list of Identities.

2. Click Register

3. Click the type of Role you want to assign to the machine, we will select “Client”

4. Click Add

5. And lastly, to save the changes, press OK.

We are now able to successfully render our 3ds Max job using Deadline Usage-Based Licensing!

But what if I have 200 machines? Do I need to register each one individually?

Of course not! We have added the ability to create “Registration Settings”, which enable you to register and assign roles to multiple machines at once. In the Manage Identities dialog, select the Registration Settings tab. 

Suppose I have 200 Render nodes with IP Addresses in the range 172.32.56.0-172.32.56.200 that I want to register and assign a Client role. To do this I would:

1. Click “Add” to create a new registration setting, 

2. Give the setting a name

3.  Set “Enabled” to True

4. Set the “Client Filter Type” to “IP Regex”

5. Enter a regular expression that matches out desired IP range, for this example that would be 172.32.56.*

6. Set the “Default Status” to Registered

7. Set the “Default Role” to Client

8. Click Ok

And voila! We have now created a registration setting that will automatically register and assign client roles to our 200 render nodes upon their first time connecting to the Deadline RCS. After all of your machines have been registered (which you can verify in the :Assign Status and Role“ tab) we recommend disabling the registration setting (Set “Enabled” to False) as to avoid inadvertently registering additional machines.

Conclusion

Deadline’s new Secret Management feature provides exciting functionality for all customers that want fined grained control and secure transmission of sensitive data on their render farms.  For more information on the new Secrets Management feature, see the official documentation.