Using Web Service in Deadline

Blog

Apr 29, 2016

Version: 8.0

WHERE DID THE WEB SERVICE GO?

When the Deadline 8.0 beta started some users expressed surprise when they were unable to use the Web Service through Pulse. These users may have thought to themselves, “Did I do something wrong?” or “Is this my fault somehow?”. The answer to both of these questions is obviously no: while Pulse and the Web Service still love YOU very much, they no longer want to be together. Pulse and the Web Service have gotten a divorce and it’s not your fault.

At a glance it may seem like Pulse got the better end of the deal in retaining its user interface and icon and that the Web Service got kicked to the curb. The truth is that the Web Service has never been happier, it is now lightweight and carefree. It no longer shares resources with Pulse. Independence is great for the Web Service.

BUT I HATE CHANGE, TELL ME HOW THIS IS GOOD FOR ME

Fair enough. Having the Web Service separate from Pulse prevents issues that affect one application from harming the other. If the Web Service is brought down for any reason, be it heavy network traffic or some other unexpected error, Pulse will not automatically suffer the same fate. Since Pulse is meant to be an administrative tool that manages, repairs and maintains the farm it would be very unfortunate if it crashed due to a problem with the Web Service. There is also the fact that the Web Service is an application that can communicate with the outside world, whereas Pulse should not be so accessible.

HOW DO I USE THE WEB SERVICE NOW?

The Web Service is a very straightforward application to set up and run. All of the configurable settings can be found in the Monitor through Configure Repository Options -> Web Service and Proxy Server. If the Web Service is running when any of these settings are modified it will need to be restarted before they will take effect.

The Listening Port is the port which the Web Service will use to receive requests. Once you have set the Listening Port, the Web Service can be reached using it’s host name or IP address and the Listening Port, like this:

http://localhost:8082/

Note that in this example the Web Service is on the same machine as the requestor and the Listening Port is 8082. If you were accessing the Web Service from another machine, you would use its host name or IP address instead of localhost. If you try to reach the Web Service this way you should receive the following response if everything is configured correctly:

This is the Deadline Web Service!

The Connection Limit is used to configure the number of simultaneous connections the Web Service can handle at once. This value is used in conjunction with the Connection Timeout value to help prevent requests to the Web Service from timing out.

The Web Service supports authentication, and the credentials used for this authentication are the Deadline user’s name, and the password that they set in their User Settings (which can be accessed from the Deadline Monitor).Require Authentication is used to enable authentication, and Allow Empty Passwords allows users with no password to access the Web Service by providing just their user name.

Allow Execution of Non-Script Commands allows Deadline Command commands to be used through the Web Service. To call Deadline Command through the Web Service you need only append the name of the command to the end of you Web Service URL. Here’s how you would call the GetVersion command:

http://localhost:8082/GetVersion

Note that this differs from how the REST API is called, as all calls to the rest api start with “api/”. Running the same command through the REST API looks like this:

http://localhost:8082/api/repository?Version=true

To run the Web Service on a Windows machine you must first reserve the namespace for the Web Service. If the namespace is not properly reserved you will see an error message like this when starting up the Web Service:

Once you have configured the Web Service it can be run as a command line application. Simply run the deadlinewebservice.exe located in your Deadline installation directory. When the Web Service is running it will look like this:

Once you have the Web Service running, you can communicate with it in the exact same way as you did when the Web Service was part of Pulse.

RUNNING SCRIPTS THROUGH THE WEB SERVICE

Scripts can be run through the Web Service, as long as they are located in the Web Service script folder in the repository. Running these scripts functions exactly the same as when the Web Service was part of Pulse. Here’s an example of how to run the GetFarmStatisticsEx script, which ships with Deadline, through the Web Service:

http://localhost:8082/GetFarmStatisticsEx

Output from the function should resemble this:

Repository time: 04/27/2016 11:11:38
Queued Jobs: 0
Rendering Jobs: 0
Errored Jobs: 0
Suspended Jobs: 1
Pending Jobs: 0
Completed Jobs: 1
Deleted Jobs: 0
FailedJobs: 0
Corrupt Jobs: 0
Rendering Machines: 0
Idle Machines: 0
Stalled Machines: 0
Offline Machines: 5
Disabled Machines: 0
[5717dbc84f0bde43843cadd5]
AuxiliarySubmissionFileNames=
Comment=
CompletedDateTime=Apr 26/1609:32:24
ConcurrentTasks=1
Department=
JobDependencies=
JobDependencyPercentage=-1
ErrorReports=0
FramesList=0
Group=none
JobId=5717dbc84f0bde43843cadd5
LimitGroups=
LimitTasksToNumberOfCpus=True
MachineLimit=0
MachineLimitProgress=-1
WhitelistFlag=False
ListedSlaves=
Name=Untitled
NotificationEmails=
NotificationTargets=ryan
OnJobComplete=Nothing
OnTaskTimeout=Error
OutputDirectories=
OutputFileNames=
OverrideNotificationMethod=False
PluginName=Python
Pool=none
PostJobScript=
PostTaskScript=
PreJobScript=
PreTaskScript=
Priority=25
ReloadRenderer=False
SequentialJobFlag=False
SlaveTimeoutSeconds=0
StartedDateTime=Apr 20/1614:43:28
Status=Completed
SubmitDateTime=Apr 20/1614:43:04
SubmitMachineName=node_01
TaskCount=1
TasksQueued=0
TasksRendering=0
TasksSuspended=0
TasksCompleted=1
TasksFailed=0
UserName=ryan
[5717dd6e4f0bde43843cade9]
AuxiliarySubmissionFileNames=
Comment=
CompletedDateTime=Jan 01/0100:00:00
ConcurrentTasks=1
Department=
JobDependencies=
JobDependencyPercentage=-1
ErrorReports=0
FramesList=0-99
Group=none
JobId=5717dd6e4f0bde43843cade9
LimitGroups=
LimitTasksToNumberOfCpus=True
MachineLimit=0
MachineLimitProgress=-1
WhitelistFlag=False
ListedSlaves=
Name=Untitled
NotificationEmails=
NotificationTargets=ryan
OnJobComplete=Nothing
OnTaskTimeout=Error
OutputDirectories=
OutputFileNames=
OverrideNotificationMethod=False
PluginName=Python
Pool=none
PostJobScript=
PostTaskScript=
PreJobScript=
PreTaskScript=
Priority=99
ReloadRenderer=False
SequentialJobFlag=False
SlaveTimeoutSeconds=0
StartedDateTime=Apr 20/1614:53:38
Status=Suspended
SubmitDateTime=Apr 20/1614:50:06
SubmitMachineName=node_01
TaskCount=100
TasksQueued=0
TasksRendering=0
TasksSuspended=50
TasksCompleted=50
TasksFailed=0
UserName=ryan

More information on Web Service scripts can be found here.

USING THE WEB SERVICE’S REST API

Using the Web Service’s REST API has not changed since separating the Web Service from Pulse. The Web Service’s REST API can be used to manage your render queue and update your farm’s settings. The REST API offers access to the Deadline objects in the form JSON dictionaries. Jobs, Worker , Reports, Repository Settings and much more can be manipulated. Here is an example of how to retrieve all the names of the Workers in the farm from a Web Service at 192.168.8.8 on port 8082:

http://192.168.8.8:8082/api/slaves?NamesOnly=true

Which will return a result like this:

["node_01","node_02","node_03","node_04","node_05"]

Web Service commands that only retrieve data use the GET request type, and pass all the necessary arguments right in the URL. When using POST or PUT requests you will need to modify the message body and request type of your request. The Web Service expects the message body contents to be made up of a single JSON dictionary that contains all the required arguments. When building up a request, it is required that you include the command you wish to execute, followed by the arguments that command needs. How do we determine what format that command requires?

Say you want to update a job’s priority, but you’re not sure what to send to the Web Service in terms of a job. The easiest way to determine this is to first request the job, and use the result as a template. For the following examples I will be using Fiddler to create my requests. Here’s how you retrieve a job using the REST API:

GET http://localhost:8082/api/jobs?JobId=5717dbc84f0bde43843cadd5 HTTP/1.1
User-Agent: Fiddler
Host: localhost:8082
Content-Length: 0

The output of this request is:

HTTP/1.1 200 OK
Content-Length: 2623
Server: Microsoft-HTTPAPI/2.0
Access-Control-Allow-Origin: *
Date: Wed, 27 Apr 2016 18:21:56 GMT

[{"Props":{"Name":"Untitled","Batch":"","User":"ryan","Region":"","Cmmt":"","Dept":"","Frames":"0","Chunk":1,"Tasks":1,"Grp"
:"none","Pool":"none","SecPool":"","Pri":25,"ReqAss":[],"ScrDep":[],"Conc":1,"ConcLimt":true,"AuxSync":false,"Int":false,"IntPer":100,"RemTmT":0,"Seq":false,"Reload":false,"NoEvnt
":false,"OnComp":2,"Protect":false,"AutoTime":false,"TimeScrpt":false,"MinTime":0,"MaxTime":0,"Timeout":1,"FrameTi
meout":false,"StartTime":0,"Dep":[],"DepFrame":false,"DepFrameStart":0,"DepFrameEnd":0,"DepComp":true,"DepDel":false,"DepFail":false,"DepPer":-1.0,
"NoBad":false,"OverAutoClean":false,"OverClean":false,"OverCleanDays":0,"OverCleanType":1,"JobFailOvr":false,"JobF
ailErr":0,"TskFailOvr":false,"TskFailErr":0,"SndWarn":true,"NotOvr":false,"SndEmail":false,"SndPopup":false,"NotEm
ail":[],"NotUser":["ryan"],"NotNote":"","Limits":[],"ListedSlaves":[],"White":false,"MachLmt":0,"MachLmtProg":-1.0,"PrJobScrp":"","PoJobScrp":"","PrTskScrp":"","PoTskScrp":"","Schd"
:0,"SchdDays":1,"SchdDate":"2016-04-20T14:43:04.954-05:00","SchdStop":"0001-01-01T00:00:00-06:00","MonStart":"-10675199.02:48:05.4775808","MonStop":"-10675199.02:48:05.4775808","TueStart":"-10675199.02:48:
05.4775808","TueStop":"-10675199.02:48:05.4775808","WedStart":"-10675199.02:48:05.4775808","WedStop":"-10675199.02
:48:05.4775808","ThuStart":"-10675199.02:48:05.4775808","ThuStop":"-10675199.02:48:05.4775808","FriStart":"-106751
99.02:48:05.4775808","FriStop":"-10675199.02:48:05.4775808","SatStart":"-10675199.02:48:05.4775808","SatStop":"-10
675199.02:48:05.4775808","SunStart":"-10675199.02:48:05.4775808","SunStop":"-10675199.02:48:05.4775808","PlugInfo"
:{"Version":"2.7","ScriptFile":"C:/Users/Ryan/test.py","Arguments":""},"Env":{},"EnvOnly":false,"PlugDir":"","EventDir":"","OptIns":{},"EventOI":[],"Ex0":"","Ex1":"","Ex2":"","Ex3":"","Ex4":"","Ex5":"","Ex6":"","Ex7":"","Ex8":"","Ex9":"","ExDic":{},"OvrTaskEINames":false,"TaskEx0":"","TaskEx1":"","TaskEx2":"","TaskEx3":"","TaskEx4":"","TaskEx5":"","TaskEx6":
"","TaskEx7":"","TaskEx8":"","TaskEx9":""},"FramesList":"0","ComFra":1,"TotRenT":"00:00:01.9085582","IsSub":true,"
Mach":"mobile-033","Date":"2016-04-20T14:43:04.964-05:00","DateStart":"2016-04-20T14:43:28.938-05:00","DateComp":"2016-04-26T09:32:24.989-05:00","Plug":"Python","OutDir":[],"OutFile":[],"TileFile":[],"Main":false,"MainStart":0,"MainEnd":0,"Tile":false,"TileFrame":0,"TileCount":0,"TileX":0,"TileY":0,"Stat":3,"A
ux":[],"Bad":[],"CompletedChunks":1,"QueuedChunks":0,"SuspendedChunks":0,"RenderingChunks":0,"FailedChunks":0,"PendingChunks":0
,"SnglTskPrg":"0 %","Errs":0,"DataSize":-1,"_id":"5717dbc84f0bde43843cadd5"}]

The GET request returns a list of requested jobs. We only requested one job so we got a JSON list (in square brackets []) that contains a JSON dictionary (in curly brackets {}). The item we are interested in is the JSON dictionary, as it corresponds to a Deadline job. The value we want to modify is its priority, which is currently set to 25 (“Pri”:25). To modify that value, we will need to use a PUT request, and populate the message body. The first entry in our message body JSON dictionary is the command we wish to perform. The command for saving a job’s properties is “save”. The next and final argument we need is the modified job itself.

PUT http://localhost:8082/api/jobs HTTP/1.1
User-Agent: Fiddler
Host: localhost:8082
Content-Length: 2647

{"Command":"save", "Job":{"Props":{"Name":"Untitled","Batch":"","User":"ryan","Region":"","Cmmt":"","Dept":"","Frames":"0","Chunk":1,"Tasks":1,"Grp"
:"none","Pool":"none","SecPool":"","Pri":50,"ReqAss":[],"ScrDep":[],"Conc":1,"ConcLimt":true,"AuxSync":false,"Int":false,"IntPer":100,"RemTmT":0,"Seq":false,"Reload":false,"NoEvnt
":false,"OnComp":2,"Protect":false,"AutoTime":false,"TimeScrpt":false,"MinTime":0,"MaxTime":0,"Timeout":1,"FrameTi
meout":false,"StartTime":0,"Dep":[],"DepFrame":false,"DepFrameStart":0,"DepFrameEnd":0,"DepComp":true,"DepDel":false,"DepFail":false,"DepPer":-1.0,
"NoBad":false,"OverAutoClean":false,"OverClean":false,"OverCleanDays":0,"OverCleanType":1,"JobFailOvr":false,"JobF
ailErr":0,"TskFailOvr":false,"TskFailErr":0,"SndWarn":true,"NotOvr":false,"SndEmail":false,"SndPopup":false,"NotEm
ail":[],"NotUser":["ryan"],"NotNote":"","Limits":[],"ListedSlaves":[],"White":false,"MachLmt":0,"MachLmtProg":-1.0,"PrJobScrp":"","PoJobScrp":"","PrTskScrp":"","PoTskScrp":"","Schd"
:0,"SchdDays":1,"SchdDate":"2016-04-20T14:43:04.954-05:00","SchdStop":"0001-01-01T00:00:00-06:00","MonStart":"-10675199.02:48:05.4775808","MonStop":"-10675199.02:48:05.4775808","TueStart":"-10675199.02:48:
05.4775808","TueStop":"-10675199.02:48:05.4775808","WedStart":"-10675199.02:48:05.4775808","WedStop":"-10675199.02
:48:05.4775808","ThuStart":"-10675199.02:48:05.4775808","ThuStop":"-10675199.02:48:05.4775808","FriStart":"-106751
99.02:48:05.4775808","FriStop":"-10675199.02:48:05.4775808","SatStart":"-10675199.02:48:05.4775808","SatStop":"-10
675199.02:48:05.4775808","SunStart":"-10675199.02:48:05.4775808","SunStop":"-10675199.02:48:05.4775808","PlugInfo"
:{"Version":"2.7","ScriptFile":"C:/Users/Ryan/test.py","Arguments":""},"Env":{},"EnvOnly":false,"PlugDir":"","EventDir":"","OptIns":{},"EventOI":[],"Ex0":"","Ex1":"","Ex2":"","Ex3":"","Ex4":"","Ex5":"","Ex6":"","Ex7":"","Ex8":"","Ex9":"","ExDic":{},"OvrTaskEINames":false,"TaskEx0":"","TaskEx1":"","TaskEx2":"","TaskEx3":"","TaskEx4":"","TaskEx5":"","TaskEx6":
"","TaskEx7":"","TaskEx8":"","TaskEx9":""},"FramesList":"0","ComFra":1,"TotRenT":"00:00:01.9085582","IsSub":true,"
Mach":"mobile-033","Date":"2016-04-20T14:43:04.964-05:00","DateStart":"2016-04-20T14:43:28.938-05:00","DateComp":"2016-04-26T09:32:24.989-05:00","Plug":"Python","OutDir":[],"OutFile":[],"TileFile":[],"Main":false,"MainStart":0,"MainEnd":0,"Tile":false,"TileFrame":0,"TileCount":0,"TileX":0,"TileY":0,"Stat":3,"A
ux":[],"Bad":[],"CompletedChunks":1,"QueuedChunks":0,"SuspendedChunks":0,"RenderingChunks":0,"FailedChunks":0,"PendingChunks":0
,"SnglTskPrg":"0 %","Errs":0,"DataSize":-1,"_id":"5717dbc84f0bde43843cadd5"}}

If the command succeeds, the result should look like this:

HTTP/1.1 200 OK
Content-Length: 7
Server: Microsoft-HTTPAPI/2.0
Access-Control-Allow-Origin: *
Date: Wed, 27 Apr 2016 17:56:45 GMT

Success

If an error occurred, the message body will contain the error instead of “Success”.

PYTHON WRAPPER FOR THE REST API

In order to facilitate communication with the Deadline Web Service, we also ship the Standalone Python API with Deadline. To use this library, first copy the Deadline folder found in your repository under “api/python” to your site-packages folder found in your Python installation directory under “Lib/site-packages”. You can now call the Standalone Python API directly from Python. Here’s an example of use the Web Service with the Standalone Python API to retrieve a Job, edit it’s priority and resume it.

from Deadline.DeadlineConnect import DeadlineConnect

#Create the connection object, for this example the Web Service is
#running on the same machine as this script

webServiceConnection = DeadlineCon( 'localhost', 8082 )

#Retrieve the job
jobId = "5717dbc84f0bde43843cadd5"
job = webServiceConnection.Jobs.GetJob(jobId)

#Modify the jobs priority

job["Props"]["Pri"] = 75

#We've modified the job's priority, but modifying its status here would not change anything. Only keys contained in 
the "Props" sub-dictionary can be modified

webServiceConnection.Jobs.SaveJob(job)

result = webServiceConnection.Jobs.RequeueJob(jobId)

#It's important to check the results commands to make sure they completed successfully. Make sure you check what 
the possible returns of your command
#is before executing it. Here we expect a message explaining whether it has succeeded or failed

print "Result of job requeue: " + result

Note that when retrieving Deadline objects (such as Jobs, Worker Info, Tasks, etc.) not all properties of these objects can be modified. For Jobs, the only properties that can be modified when calling SaveJob are properties found in the sub-dictionary “Props”. Other examples of values that cannot be changed include but are not limited to: job IDs, worker IDs, task IDs, etc.

If you are attempting to modify an object’s property and are unable to do so, check if there is an API command for modifying that property specifically. If none can be found, and modifying the property and then saving does not update the object, please contact support.

WRAP UP

The Web Service may be separated from Pulse, but that does not change its effectiveness or even how to use it. The Web Service can be used to execute scripts remotely, access and modify data on the farm much like Deadline Command, or even run Deadline Command itself.