Global Asset Libraries for Next-Generation Animation Workflows
Summary
DreamWorks Animation has created and deployed a new DreamWorks Working Asset Library (the frontDesk Global Library) system that implements elements of the MovieLabs® 2030 Vision. DWA maintains a global library of models and 3D assets as a single source of truth in cloud-based storage, automating access and authorizations, and organizing assets in the library through a standardized workflow ontology. The frontDesk Global Library is built on top of DreamWorks’ comprehensive animation pipeline.
The frontDesk Global Library relies on custom-built middleware that serves as an abstraction layer between users, applications, and assets. DWA integrates artist applications with the studio middleware, enabling productions to use the applications in conjunction with the frontDesk Global Library. The system relies on centralized cloud storage for creative assets, with all users syncing their work to one set of common DWA assets that are organized according to a standardized ontology of data, assets, and workflow relationships, kept current with a single source of truth for the most current approved assets. The system supports on-site and remote users with a log-in that assigns rights and access based on the relevant production and workflow requirements. The frontDesk system aligns with the MovieLabs 2030 Vision by relying on a hybrid cloud to deliver a centralized production resource that streamlines creation and reuse of 3D assets and replaces more time-consuming manual processes with online search-and-retrieval access.
Background
Creation of an animated feature involves managing a large team of contributors engaged in a constant flow of artwork, modeling, surfacing, rigging and animating across every frame of the feature in a continuous iteration cycle. The asset and workflow management challenge is difficult, even with a fully co-located team working on one connected network at a single facility; it becomes exponentially harder with work distributed around the world or with a pandemic forcing the team to work remotely. Sharing the work across different locations and artists is a challenge, as it is almost impossible to know exactly which files any particular artist needs to access at any given time. The usual response was to share a copy of the entire traditional file tree (or all the branches of the tree relevant to a particular department’s work), listing the hierarchy and location of all folders and files, then attempting to keep all versions of the file tree in sync. The files were huge and time-consuming to send. The syncing process was complex. The sheer volume of the data (up to 1 PetaByte) made it extremely difficult to keep the tree in sync over time. There were situations where artists submitted finished animation work based on older models rather than the current version – days or weeks of work might be thrown away.
To maintain continuity between productions and streamline new projects, artists also needed ongoing access to an archive of models, artwork, and animation assets from previous productions. Retrieving work from storage for reuse in new productions was a burdensome, time-consuming, and recurring task across almost all production workflows. Finding relevant files involved searching across multiple fixed locations or arbitrary storage places on disks at the studio, then manually pulling identified material back into a more accessible location for use by the production. The process created the risk that valuable files identified for reuse might be lost when a production shut down or when disk resources were cleared out from storage, or that artists would choose to rebuild assets because the search process was too cumbersome.
The task of providing artists with constantly updated access to both work-in-progress and archival working assets has always been extremely challenging. With the increasingly ambitious scope of productions, keeping all users synced on the current versions became unsustainable and unscalable.
DWA knew it needed to be smarter about handling large volumes of files and providing secure access to all users, with better lifecycle management of both the file tree and all the individual assets. All contributors needed guaranteed access to the latest files, connectivity that enabled real-time collaboration and iteration, and access to the most current “approved by director” version, so all participants stayed in sync.
The assets shared among the productions teams also needed to be organized in a commonly understood structure. The structure needed to have a core logic and ontology, while allowing users to adapt the system for their own purposes and add their own descriptions and metadata.
Solution
The goal of the frontDesk Global Library was to address the shortcomings of the legacy animation workflow by:
- Building a new centralized file system – Replacing the distributed tree system of file-sharing with one central repository of models and 3D assets, maintained in cloud-based storage with comprehensive real-time version control that ensures all contributors have the most current assets with notice of updates.
- Expanding access to the working asset archive – Transferring most current versions of models from offline storage to the new frontDesk Global Library storage system. Enabling artists and other authorized users to search for relevant assets, receive search results more quickly in the form of cloud addresses, and either pull assets into current project folders and applications or work with files streamed from the cloud address.
- Leveraging a common ontology and organizational structure of assets – Deploying a well-understood structure for the shared assets so that all users could rely on some common definitions and relationships, while also being able to augment the shared ontology with specific user or department descriptions and metadata.
- Secure provisioning – For secure production assets in the frontDesk Global Library, giving access only to those portions of the Library relevant to the user’s current project and prior related productions. That approach aligns with the MovieLabs CSAP (Common Security Architecture for Production) notion of “least privilege” and also minimizes clutter for end users who no longer need to sift through assets not required for their work. After log-in, each user is automatically assigned access rights based on their current project and role.
Architecture
A key component of the new system is abstraction middleware running on servers in DreamWorks’ hybrid cloud. An API connects the middleware to applications and asset storage. Users log into the system and are connected to their asset storage tools based on their access privileges.
Artist log-ins are linked to user roles and profiles that deliver predetermined access to the Global Library based on the role of the user in the pipeline workflow. Users are able to download assets for local editing or work with the files streamed directly from the frontDesk Global Library.
Assets are maintained in a hybrid-cloud storage architecture, that includes on-prem and cloud resources. The location of the assets is largely invisible to users, and artists often do not know where their assets are stored and do not need to know, as long as they are available rapidly for tasks the artist needs to complete. Assets are organized according to a standardized ontology, which also enables metadata about the contents of files and the semantic relationships between files. These ontology definitions became key contributions to the open MovieLabs Ontology for Media Creation.
Some users access the frontDesk Global Library on a local network through workstations on the lot while most remote users rely on virtual desktop infrastructure (VDI). Initially, there was concern that users would experience latency and degraded user experience via these remote connections. However, the realities of COVID served as a strong motivator encouraging users to embrace the remote experience in exchange for the ability to work from home or from other distant locations.
“frontDesk is a monumental leap in transforming productivity and library management.”
BENEFITS
The frontDesk Global Library has been used in the recent production of Trolls Band Together, with a more expanded Digital Backlot vision for upcoming productions. The Global Library concept has received positive reviews from the production teams, who have noted the reduced time and fewer steps involved in accessing character previews, models, and assets available for reuse. Users also appreciate the visually rich interfaces and the easy search capability.
The system has enabled DWA to be smarter about costs. A centralized system has eliminated duplication of effort and resources. The frontDesk system helps make the pipeline more flexible and extensible to take advantage of the most efficient resources wherever they are by giving independent and remote workers easy access to the most current and reusable working assets.
“For finding and retrieving model assets, frontDesk is a huge shift in efficiency compared to how we used to do things (using a File->open browser). If I had to go back to doing things the old way, it would feel like going from a Ferrari to a skateboard.”
Alignment with MovieLabs 2030 Vision Principles
PRINCIPLE 1
The frontDesk Global Library maintains all current models and many reusable archive assets in a centralized hybrid cloud. The cloud system provides a single source of truth for all approved assets available for reuse from current and historical productions.
PRINCIPLE 3
The frontDesk system “publishes” assets to users and artists as part of the workflow process. Users in the workflow are notified when assets or tasks are ready for handoff to them. For current production assets, users receive cloud addresses for relevant assets and can choose whether to work directly with the cloud files streamed to their desktop environment or download the files for local editing. For archival assets from prior productions, the system provides a copy that is added to a current project and mapped into the current production workflow.
PRINCIPLE 4
The frontDesk Global Library gives users access to archives of the most current approved materials from the same production and previous completed productions (such as previous titles in a trilogy). Users are given access rights and permissions to search and retrieve materials from their current production and related prior productions. Users no longer have to search for materials across storage discs from multiple different productions, trolling directories to find assets for reuse. Instead, all up-to-date materials approved for reuse are centralized in one archival database, without risk that materials could be deleted or made inaccessible when separate production storage discs are cleaned up or moved.
PRINCIPLE 8
Assets are organized, described, and interrelated using a standard ontology. The standard ontology makes cross-referencing and search more efficient since a core of shared standard definitions are used for all productions.
Lessons Learned
DreamWorks learned a key lesson around building a standard ontology as the foundation for the asset storage and description system. Working on this project, DWA gained extensive experience with its ontology and toolset, and it has worked closely with MovieLabs to incorporate key learnings into the MovieLabs Ontology for Media Creation (OMC). DWA will continue working with MovieLabs to share learnings and tools with the community in the hope of everyone working together to build a truly global ontology. That would be a benefit to the entire industry, enabling increased compatibility across tools and databases and eliminating duplicative and wasted effort translating between metadata structures.
DreamWorks also learned the importance of change management when new systems are introduced. It is best to involve creatives and other production users early and solicit their input and feedback throughout the process, as well as maintain sufficient flexibility to allow users to contribute to defining their own pipeline workflows and the structuring of data. For example, a key element of the success of the frontDesk Global Library was the decision to build it based on a structure created by the artists themselves.
A true cross-platform Production User ID (PUID) capability would make all pipeline systems more efficient and the user experience more customized and seamless. A continuing challenge is the need to navigate separate role management systems within different applications that interact with assets from the frontDesk system. There still is no multi-platform log-in experience that carries identity and role management across applications and platforms. The industry needs more compatibility and accessibility across applications
NEXT STEPS
Although the system is smarter about what gets moved and how it is moved and managed, the frontDesk Global Library still moves data from cloud-based storage to users on an as-needed basis. In the future, the goal is to move less data and keep more assets permanently in the cloud, with potential improvements in the ability to edit or access files directly in the cloud without a degraded user experience. The team also plans to bring more asset types into a Global Library system and to expand the reach of the system via additional cloud resources.
In addition, DreamWorks is also planning to take more steps to implement the zero-trust model of MovieLabs CSAP (Common Security Architecture for Production). Today, users sometimes have more access than they need, which creates some inefficiencies and possible security concerns. Those risks have to be balanced against the need to get work done in a flexible way and provide artists with access to related libraries and assets from previous titles. As workflows and pipeline requirements become better defined, it may be possible to reduce those risks and take advantage of additional efficiencies.
Other improvements planned by DWA include:
- Expanding the scope of the repository (beyond the latest approved assets for models) by adding more categories of materials so that the frontDesk system becomes a true “digital backlot” available to all productions
- Implementing full version control to track artist refinements and improvements to frontDesk assets
- Providing notifications to users when original assets change in the Global Library
- Working with MovieLabs to synchronize the DWA standard ontology with the MovieLabs common ontology
- Expanding use of the DWA production user ID to include managing vendor users with the same common production ID
By taking advantage of the more constrained environments of an animation pipeline, DreamWorks has been able to test and incubate some of the key concepts that are part of the MovieLabs 2030 Vision. DreamWorks is optimistic that its experiences and learnings will transfer to live action productions and that industry alignment around the 2030 Vision will create opportunities for additional innovation and further enhancements to the efficiency of the creative process.
MOVIELABS PERSPECTIVE
The DreamWorks frontDesk Global Library demonstrates the true value of the 2030 Vision to centralize assets in the cloud and then enable greater efficiency and time-savings across the entire creative team. The Global Library that DWA has created embraces a constantly changing world by accommodating both on-site and remote workers with one central system that keeps all contributors in sync and working on the most current assets. The work so far completed is foundational and will enable more flexibility to expand in the future as pipelines continue to evolve and new technologies are adopted. We look forward to a broader ecosystem where internal platforms like DWA’s can securely interoperate with external systems (upstream, downstream and vendors) to seamlessly pass notifications, asset identifiers and metadata – moving the industry to true global, multi-system collaboration.
Get the Case Study
Download a free PDF of this case study