Asset Caching in Autodesk Maya using Deadline

  • Blog

Version: Deadline 8.1

INTRODUCTION

Deadline 8.1 includes many exciting new features and enhancements for users of Autodesk Maya. "Local Asset Caching" is a one of the new features we are most proud of and it's already receiving rave reviews by studios.

LOCAL ASSET CACHING SAVES TIME AND MONEY

Deadline's Maya plugin now includes a feature that can save you time and money by significantly reducing network traffic when rendering Maya scenes. Reducing network traffic can often shave off several minutes of render time per frame. The result of Local Asset Caching is faster renders, less worker compute time, quicker turnarounds, and less stress on network infrastructure.

MAYA RENDERS AND NETWORK ASSETS

When rendering using Maya over Deadline, often there are many additional files that are needed for a render. For example, texture files, caches, and simulation data. These files are most often placed in network storage and accessed at render time. These files can be large, ranging from several megabytes to in excess of a gigabyte. The render time of a frame is heavily dependent on the network access speed and reliability. In extreme cases, the renderer might spend more time accessing data over a network than it does rendering.

LOCAL CACHES ON DEADLINE WORKERS

Local asset caching stores a local copy of all assets used during Maya renders. Each render then uses the assets from the local machine, rather than having to access them from their network location. Subsequent frames or new scenes submitted with the same assets will all make use of the local cache and do not need to reach out to the network. The cache itself is not tied to a particular Maya scene. It is accessible from all Maya renders.

This caching is done as a pre-process, so virtually all network traffic is eliminated during the actual Maya render. This has several advantages. If the network infrastructure experiences difficulty, it will not cause a render failure. Additionally, there are no open file handles to network paths during the rendering process.

NUTS AND BOLTS: HOW LOCAL ASSET CACHING WORKS IN MAYA

When a Maya job is picked up by a Deadline worker, the local asset caching system begins.

First, Deadline will examine the Maya scene and identify any assets that are required that are in a network location.

Once a list of assets is determined, Deadline will collect any of the assets and store them in its local cache. If the file already exists in the cache from a previous render, the file will not be cached a second time. If the network asset has been modified since the last time it was cached, Deadline will invalidate the cache, and the file will be updated locally.

At this point, the cache contains all the remote assets that are needed for the render. Deadline now modifies the Maya scene file. All textures, simulation data, caches, and any other network paths are now changed to point at the local cache. Deadline does not modify the original scene file. The repathing is done at render time leaving the Maya project unchanged.

Deadline then hands controls over to the renderer. Once render begins, all assets will be accessed from the local cache, and the render is no longer dependent on any network data.

As a final step in the process, Deadline performs a cleanup of its local asset cache. This keeps the cache from eating up too much storage space on the workers and removes old, unused assets.

CONFIGURING THE REPOSITORY TO USE LOCAL ASSET CACHING

Setting up Local Asset Caching in Maya is incredibly easy. Here are some basic instructions to get you going:

All the options for configuring Maya local asset caching are available through the Deadline Monitor under Tools > Configure Plugins > MayaBatch. The controls are straightforward and easy to use. The following are the list of options available to users, and what they do:

  • Remote Asset Path: When scanning a Maya scene file, it is necessary to identify which assets are located on a remote storage device. The way Deadline determines that is by checking its path prefix. If the path starts with a prefix listed under this "Remote Asset Path" option, it is assumed to be on a remote storage device, and will be cached locally upon render.
  • Worker LAC Directory: These options determine where on the local worker drive the assets will be cached. Since directory structures are set up differently across different platforms, and caching is cross-platform, it is necessary to have three different options based on the operating system that the slave is running on.
  • Days until cache delete: It is necessary to do some housekeeping on the workers in order to clean up the local cache and prevent the local storage to fill up unnecessarily. This option allows user to specify when an asset in the cache will be deleted. If a cached asset is not accessed within the specified number of days, that file will be removed from the cache.

TRY IT OUT! SUBMIT A MAYA SCENE USING LOCAL ASSET CACHING

Using local asset caching is as easy as clicking a check box. Open the integrated Maya submitter, and enable "Use Local Asset Caching" under the Render Options rollout, and you're good to go. This feature is only available when using "Maya Batch" mode, so be sure to enable "Use MayaBatch Plugin" in the submitter as well.

WRAPPING UP

Deadline's 2017 New Year's resolution is to make network overhead a thing of the past. You can help us achieve our goal by enabling Local Asset Caching on your Maya renders and providing us feedback. Let us know what you think of Local Asset Caching on the Deadline beta forums. Once Deadline 8.1 is released, you can use the regular Deadline forums for feedback.