Custom MaxScript Submission Tutorial
The following Tutorial is intended for technical artists and technical directors and demonstrates the basics of submitting jobs to Deadline using MAXScript, the built-in language of Autodesk 3ds Max.
Deadline provides two main ways for submitting Deadline jobs from Autodesk 3ds Max - via the Monitor submission scripts and via the integrated "Submit Max To Deadline" (short: "SMTD") script from inside the application.
The SMTD script is significantly more advanced than its Monitor counterpart and allows the submission of either the currently loaded 3ds Max scene as a Render job, or of a custom MAXScript as a script (processing) job, but there are some workflows that are still not covered by it - for example the submission of a scene that is already saved on disk and is NOT currently loaded in 3ds Max (something the Monitor submission script does), or the batch-submission of an arbitrary number of 3ds Max scene files saved on disk (which no script shipping with Deadline currently does).
Thankfully, the SMTD script was implemented as a set of library functions and a separate User Interface script, so it is relatively easy to access the majority of its features by simply loading and calling these libraries.
In the next section, we will take a look at the simplest possible submission script to demonstrate the very basics of the communication with Deadline from inside of 3ds Max. Then, we will use the basic script to add more features and will round it off as a small MacroScript utility with its own User Interface.
SIMPLEST 3DS MAX SUBMITTER
Let's write a very simplified version of a 3ds Max job submitter. We will use it as the basic structure for our later enhancements.
The following is the complete basic submitter code. We will discuss it line-by-line below:
( global SMTDSettings global SMTDFunctions local theNetworkRoot = @"\\GATEWAY\DeadlineRepository" local remoteScript = theNetworkRoot + @"\submission\3dsmax\main\SubmitMaxToDeadline_Functions.ms" fileIn remoteScript SMTDFunctions.loadSettings() SMTDSettings.JobName = maxFileName + " [SIMPLE MXS SUBMISSION]" local maxFileToSubmit = SMTDPaths.tempdir + maxFileName SMTDFunctions.SaveMaxFileCopy maxFileToSubmit local SubmitInfoFile = SMTDPaths.tempdir + "\\max_submit_info.job" local JobInfoFile = SMTDPaths.tempdir+ "\\max_job_info.job" SMTDFunctions.CreateSubmitInfoFile SubmitInfoFile SMTDFunctions.CreateJobInfoFile JobInfoFile local initialArgs="\""+SubmitInfoFile+"\" \""+JobInfoFile+"\" \""+maxFileToSubmit+"\" " SMTDFunctions.waitForCommandToComplete initialArgs SMTDSettings.TimeoutSubmission )--end script
We start by opening and closing parentheses to define a local scope for our script.
We will be using the SMTD function library, as well as the SMTD settings which are based on factory defaults, user defaults and sticky settings - in other words, any settings seen in the SMTD dialog will persist and will be used in our submission unless explicitly overridden.
The SMTD Settings and the SMTD Functions are stored in global variables called, as one would expect, SMTDSettings andSMTDFunctions, respectively.We must pre-declare these variables as global in the beginning of our script to ensure the rest of the script sees them as global.
The SMTD functions are defined by a script called "SubmitMaxToDeadline_Functions.ms" which resides in the\submission\3dsmax\main\ folder of the Deadline Repository. Our first action would be to ensure it is loaded from there. Please note that in Deadline versions prior to 6.0, the files were located under \submission\3dsmax\, without the \main\ part!
To keep the script simple, we will hard-code the location of the Repository for now - on the machine this script was developed, that location was \\GATEWAY\DeadlineRepository.
PLEASE BE SURE TO REPLACE IT WITH YOUR DEADLINE REPOSITORY LOCATION!
Later on we will modify the script to detect the path automatically.
We build the name of the script to load by concatenating the NetworkRoot variable containing the location of the Repository and the folder names and script name of the Functions source. Then we call fileIn() to load that script
At this point, both the SMTDFunctions and SMTDSettings global variables are assigned MAXScript struct values - theSMTDFunctions contains a struct of functions, the SMTDSettings contains a struct instance with properties defining all submitter settings. But while the SMTDFunctions can now be called to perform various operations, the SMTDSettings are preliminarily initialized to factory defaults, as encoded in the source script. To load the defaults that the SMTD dialog would normally display, we have to call the function SMTDFunctions.LoadSettings(). This will load any global or local defaults as well as sticky settings into the struct's properties.
For example, if you had previously opened the SMTD dialog and set the Priority to 75 by sliding the colorful slider or entering a value in the Priority spinner, calling theSMTDFunctions.LoadSettings() will ensure that the default factory value of 50 is replaced with the latest sticky setting 75.
For now we won't touch any of these settings except for the Job Name which describes what the job is about. It is defined by theSMTDSettings.JobName property and we will set it to the name of the current 3ds Max scene, plus a custom suffix explaining that this job was submitted by a custom simple MAXScript submitter.
Note that you can type in show SMTDSettings in the MAXScript Listener to see the list of all available properties that are used by the SMTD Functions.
Now we want to save the current 3ds Max scene content to a temporary file to send with the job. We define a filename variablemaxFileToSubmit using the pre-defined temporary path location of SMTD stored in the property SMTDPaths.tempdir and the current scene filename. The struct SMTDPaths was defined as a global variable by the Functions script. We could use the 3ds Max temporary path returned by getDir #temp instead, but since the 3ds Max Submitter has already provided good locations for most of its files, let's use them.
We call the function SMTDFunctions.SaveMaxFileCopy() and pass the temporary path variable as argumemt. The current scene will be saved to disk at the specified location without affecting the actual file name and path of the current 3ds Max session.
Next on the list of things to do is the creation of the two INI files that control the job submission and the job processing, respectively. We define two variables in the same temporary directory as used for the 3ds Max file, then we call the functionsSMTDFunctions.createSubmitInfoFile() and SMTDFunctions.createJobInfoFile() with the two file names as arguments to generate the data.
The content of the Submit Info file will be used to determine the parameters of the submission process. In Deadline 5.x and earlier, the information from this file was used to generate a new XML file that was stored in the job's folder in the Repository. In Deadline 6 and higher, the data is stored in the database.
The content of the Job Info file is used to control the actual rendering. Some of these values are used by the Job Plugin, others are used to override the renderer's settings. You can provide ANY keys and values in this file to store custom information, and read those values using MAXScripts running as part of the job to pass arbitrary data from the submitter to the job.
The creation of the two control files will ensure that all relevant rendering settings like output filename, render range, renderer settings etc. as well as all SMTD settings like Priority, Job Name etc. are passed to the job.
Now all that's left to do is sending the two control files plus the 3ds Max scene file to Deadline to start rendering. To do this, we define a command line variable called initArgs and concatename the file names of the Submit Info file, the Job Info file and the 3ds Max file enclosed in quotation marks (to handle eventual spaces in the file names), separated by a space character. Then we call the function STMDFunctions.waitForCommandToComplete() and pass to it two argument s - the command line variable and the SMTDSettings.TimeoutSubmission property which defines the number of seconds to wait before assuming that the submission cannot be performed successfully.
The return value of the SMTDFunctions.waitForCommandToComplete() is a name value. It will be equal to #success if the submission was performed correctly, #failed if the submission was not successful, #readerror if there was a problem reading the result files of the command line submitter, or #timeout if the submission couldn't be performed in the time specified by the second argument.
TESTING THE SIMPLEST SUBMITTER
Now that we have the code of out basic submitter, let's see how we can test whether it is working correctly. Please keep in mind that our code has no error checking at all, although some of the SMTDFunction calls might perform some error handling internally. Thus, it is very important to prepare a VALID submission case before trying out the script.
- Start a new 3ds Max scene.
- Create a Teapot (or any other scene content)
- Add a Bend modifier, go to frame 100, enable AutoKey and set the Amount to 90.0. Disable AutoKey.
- Open the Render Setup Dialog
- Set the Time Output to Active Time Segment.
- Set the Render Output filename to a valid network location accessible to all your Render Nodes. You can use a mapped drive available on all machines, or a UNC path.
- Save the scene to disk to provide a valid name for the submitter.
At this point, you can evaluate the Simplest Submitter source code by going to Tools>Evaluate All in the MAXScript Editor or by pressing Ctrl+E. If everything went ok, you should see #success printed in the MAXScript Listener and a new job should appear in the Deadline Monitor (if you have it running).
IMPROVING THE SIMPLEST SUBMITTER
Let's take this basic code and add some more features to it. Here is the expanded code, with comments following it:
( global SMTDSettings global SMTDFunctions local theNetworkRoot = @"\\GATEWAY\DeadlineRepository" local remoteScript = theNetworkRoot + @"\submission\3dsmax\main\SubmitMaxToDeadline_Functions.ms" local localScript = getDir #userscripts + "\\SubmitMaxToDeadline_Functions.ms" if doesFileExist remoteScript do ( if SMTDFunctions == undefined do ( deleteFile localScript copyFile remoteScript localScript fileIn localScript ) SMTDFunctions.loadSettings() SMTDSettings.JobName = maxFileName + " [SIMPLE MXS SUBMISSION]" SMTDSettings.Comment = "Created using the simplest Deadline submission script imaginable." SMTDSettings.Priority = 80 SMTDSettings.ChunkSize = 10 SMTDSettings.LimitEnabled = true SMTDSettings.MachineLimit = 5 local maxFileToSubmit = SMTDPaths.tempdir + maxFileName SMTDFunctions.SaveMaxFileCopy maxFileToSubmit local SubmitInfoFile = SMTDPaths.tempdir + "\\max_submit_info.job" local JobInfoFile = SMTDPaths.tempdir + "\\max_job_info.job" SMTDFunctions.CreateSubmitInfoFile SubmitInfoFile SMTDFunctions.CreateJobInfoFile JobInfoFile local initialArgs = "\""+SubmitInfoFile+"\" \""+JobInfoFile+"\" \""+maxFileToSubmit+"\" " local result = SMTDFunctions.waitForCommandToComplete initialArgs SMTDSettings.TimeoutSubmission local renderMsg = SMTDFunctions.getRenderMessage() SMTDFunctions.getJobIDFromMessage renderMsg if result == #success then ( format "Submitted successfully as Job %.\n\n%\n\n" \ SMTDSettings.DeadlineSubmissionLastJobID renderMsg ) else format "Job Submission FAILED.\n\n%" renderMsg ) )--end script
First let's look at the loading of the SMTD Functions from the Repository. This can be slower than loading the same file from the local drive. We can do two things to improve this - check whether the SMTDFunctions variable is undefined and only then load the script, and copy the script file to a temporary folder on the local drive and then fileIn() its source from there.
It is also a good idea to only execute the rest of the script if the remote script could be found, thus avoiding an error in case the hard-coded Repository location turns out inaccessible.
Once the functions and settings are loaded, we can add more property overrides like Comment, Priority, Task Chunk Size, Machine Limit etc.
Once again, you use the expression show SMTDSettings to get the full list of all properties available. Please note that some of them are used internally, but those that reflect submission settings have quite obvious names.
For example, the property controlling the name of the pool the job will be assigned to isSMTDSettings.PoolName, while the group can be set using SMTDSettings.Group etc.
Now let's add some better result handling to the actual submission function. First, we will store the resulting name value in a variable called result.
Then we will call the function SMTDFunctions.getRenderMessage() which reads the output of the command line submitter and can be very useful when debugging problems during the submission process. We will store this message in a variable to display in the MAXScript Listener, and we can also call the function SMTDFunctions.GetJobIDFromMessage() with the message as argument to extract the Job ID of the submitted job.
Depending on whether the result was #success or not, we can now print the render message and the Job ID of the new job or just the render message containing details about the cause of the failure.
RESOLVING THE REPOSITORY LOCATION
In the previous two versions of the script, we assumed (quite reasonably) that the location of the Repository does not change often and can be hard-coded. Of course, if you want to give this script away to other people that might have a different naming scheme for their Deadline sutup, asking them to edit a hard-coded string before running the code would be unrealistic.
In the following example, we will ask Deadline for the location of its Repository folders, allowing the script to run on any network without changes:
( global SMTDSettings global SMTDFunctions local theResultFile = getDir #temp + "\\_result.txt" local theCommandLine = ("deadlinecommand.exe -getrepositoryroot > \""+theResultFile +"\"") hiddenDosCommand theCommandLine startpath:"c:\\" local theFileHandle = openFile theResultFile local theNetworkRoot= readLine theFileHandle close theFileHandle local remoteScript = theNetworkRoot + @"\submission\3dsmax\main\SubmitMaxToDeadline_Functions.ms" local localScript = getDir #userscripts + "\\SubmitMaxToDeadline_Functions.ms" if doesFileExist remoteScript do ( if SMTDFunctions == undefined do ( deleteFile localScript copyFile remoteScript localScript fileIn localScript ) ........
Let's take a look at how the above code works:
We first initialize a variable theResultFile which will hold the location of the Deadline Command output. We use the temp. folder of 3ds Max because at this point we don't have any info about the location of the Deadline temporary folder.
Next we call the hiddenDosCommand() MAXScript function to execute the deadlinecommand.exe with a command line argument -getRepositoryRoot which asks Deadline to return the location of the Repository as the result. We redirect the output of the command line execution to the temp. file we defined previously - otherwise the result would go to the console and we would not be able to acquire it. We also specify the start path of the hiddenDosCommand to be at the root of drive C:\, mostly to work around a bug in MAXScript in versions prior to 3ds Max 2010 where omitting this optional argument would cause failure.
Once the command has returned, we can assume that the temporary file contains the value we need, so we open the file, read one line out of it and store it in theNetworkRoot variable, then close the file.
If everything went ok, the rest of the script should work just like before, but the script will now adapt to the actual location of the Deadline Repository.
DEVELOPING A USER INTERFACE
So far we had to invoke the script by pressing Ctrl+E to evaluate it. While this is ok for testing and debugging purposes, a good script should make it easier for an artist without any scripting knowledge to launch and use.
Let's turn our script into a litte utility with a floating dialog, some spinners, checkboxes and buttons:
( global SMTDSettings global SMTDFunctions local theResultFile = getDir #temp + "\\_result.txt" local theCommandLine = ("deadlinecommand.exe -getrepositoryroot > \""+theResultFile +"\"") hiddenDosCommand theCommandLine startpath:"c:\\" local theFileHandle = openFile theResultFile local theNetworkRoot= readLine theFileHandle close theFileHandle global SimpleDeadlineSubmitter_Rollout try(destroyDialog SimpleDeadlineSubmitter_Rollout)catch() rollout SimpleDeadlineSubmitter_Rollout "Simple Deadline Submitter" ( spinner spn_priority "Priority:" range:[1,100,SMTDSettings.Priority] type:#integer fieldwidth:50 spinner spn_chunkSize "Frames Per Task:" range:[1,100,SMTDSettings.ChunkSize] type:#integer fieldwidth:50 checkbox chk_limitEnabled "" across:2 checked:SMTDSettings.LimitEnabled spinner spn_machineLimit "Machine Limit:" range:[1,100,SMTDSettings.MachineLimit] type:#integer fieldwidth:50 button btn_submit "SUBMIT SCENE..."width:190 height:30 align:#center on btn_submit pressed do ( local remoteScript = theNetworkRoot + @"\submission\3dsmax\main\SubmitMaxToDeadline_Functions.ms" local localScript = getDir #userscripts + "\\SubmitMaxToDeadline_Functions.ms" if doesFileExist remoteScript do ( if SMTDFunctions == undefined do ( deleteFile localScript copyFile remoteScript localScript fileIn localScript ) SMTDFunctions.loadSettings() SMTDSettings.JobName = maxFileName + " [SIMPLE MXS SUBMISSION]" SMTDSettings.Comment = "Created using the simplest Deadline submission script imaginable." SMTDSettings.Priority = spn_priority.value SMTDSettings.ChunkSize = spn_chunkSize.value SMTDSettings.LimitEnabled = chk_limitEnabled.checked SMTDSettings.MachineLimit = spn_machineLimit.value local maxFileToSubmit = SMTDPaths.tempdir + maxFileName SMTDFunctions.SaveMaxFileCopy maxFileToSubmit local SubmitInfoFile = SMTDPaths.tempdir + "\\max_submit_info.job" local JobInfoFile = SMTDPaths.tempdir+ "\\max_job_info.job" SMTDFunctions.CreateSubmitInfoFile SubmitInfoFile SMTDFunctions.CreateJobInfoFile JobInfoFile local initialArgs = "\""+SubmitInfoFile+"\" \""+JobInfoFile+"\" \""+maxFileToSubmit+"\" " local result = SMTDFunctions.waitForCommandToComplete initialArgs SMTDSettings.TimeoutSubmission local renderMsg = SMTDFunctions.getRenderMessage() SMTDFunctions.getJobIDFromMessage renderMsg if result == #success then ( format "Submitted successfully as Job %.\n\n%\n\n" \ SMTDSettings.DeadlineSubmissionLastJobID renderMsg ) else format "Job Submission FAILED.\n\n%" renderMsg )--end if )--end on button pressed )--end rollout createDialog SimpleDeadlineSubmitter_Rollout width:200 )--end script
Now let's look at the new code line by line to see what it does:
First, we introduce a new global variable SimpleDeadlineSubmitter_Rollout which will hold the rollout definition used for our floating dialog. We use a global variable to ensure that we can close a previously opened dialog before opening a new one. In fact, we attempt to close the dialog in the very next line - if the variable contains a rollout that is opened as a dialog, it will be closed, otherwise an error will be suppressed by the try()catch() error trap.
Next, we define the rollout and add several integer spinners for the Priority, Frames Per Task and Machine Limit, plus a checkbox for the enabling of the machine limit and a button to launch the submission. We initialize all these UI controls to the respective SMTDSettings which have been loaded already during the evalulation of the SMTD Functions script.
The only event handler we need is for the button which performs the submission. We include the majority of our existing script into the button's event handler. Inside the handler, we make sure the relevant SMTDSettings are set to the current values of the UI controls.
Finally, after the rollout is created, we call the function createDialog() to turn the rollout definition into a floating dialog. We specify only the width and let MAXScript figure out the necessary height. This means that adding more controls to the UI does not require changes to the dialog itself.
If we would load the same scene used in our first submission test and then press Ctrl+E to open the newly defined dialog, we can now change the Priority to, say, 62, the Frames Per Task to 25 and the Machine Limit to 2. Then we can hit the button to submit the scene to Deadline.
TURNING THE SCRIPT TO A MACROSCRIPT
We have a simple submission utility with its own User Interface, now let's turn it into a MacroScript to allow the end user to add it to the 3ds Max Menus, Toolbars, QuadMenus or Keyboard shortcuts.
We can simply provide a new first line
macroScript SimpleDLSubmit category:"Deadline" ( global SMTDSettings global SMTDFunctions ......
Evaluating the modified script by pressing Ctrl+E will appear to do nothing, but in fact it will create a new Action Item in the Customize User Interface dialog of 3ds Max.
Go to 3ds Max Main Menu > Customize > Customize User Interface and look under the "Deadline" category for the new script.
Drag this entry to a toolbar or customize a Menu, QuadMenu or Keyboard Shortcut using the typical 3ds Max workflow. Then activate the item (press the button or select the menu / press the shorcut) to open the Simple Submitter.
Using MAXScript and some of the pre-defined Submit Max To Deadline library functions, we were able to create a simple submission script for sending the current 3ds Max scene to render on Deadline.
This script does not provide any advantages over the shipping SMTD script, but it demonstrates the basic concepts of job submission and can be used as a template to more advanced tools.
In a follow-up tutorial, we will see how we can modify the script to batch-submit multiple .MAX files in one go.