How cloud-native agility helped DNEG level up its award-winning VFX

As one of the world's leading visual effects (VFX) providers for the entertainment industry, UK-based DNEG has worked on blockbuster films like Dune, Blade Runner 2049, and Oppenheimer. However, after rapid growth to over 9,000 employees across seven global sites, DNEG's legacy infrastructure was struggling to keep pace.

Software infrastructure architect Ollie Harding tells ITPro that his firm was dealing with outdated systems and tricky delays. "We've got a traditionally very monolithic architecture and that comes with all the classical problems, it takes a week to release a change ... there’s very little agility.

“Even small software changes took an extremely long time for artists to access, slowing productivity and innovation.”

It became clear DNEG needed to modernize its core infrastructure to enable the scale and agility demanded by today's fast-paced VFX industry. The company soon launched an ambitious initiative to re-platform parts of its technology stack and overhaul delivery processes.

Objectives for DNEG’s cloud-native journey

The key goals DNEG aimed to achieve included accelerating software delivery times to artists (from a week plus per release to under 10 minutes). It also wanted to move from a rigid "department of no" to a flexible "department of yes". There was also a need to future-proof architecture for innovation and next-generation demands.

"We will be able to completely revolutionize this space [in our architecture]," Harding tells ITPro.

"Now we’ve got this thing out in the corner, it’s separate: we can now start saying okay, that aspect of it, we are going to change the way we represent that asset.”

DNEG began implementing this cloud-native initiative in early 2020. The initial evaluation process considered ease of on-prem support, flexibility for future growth, and depth of vendor expertise as key criteria.

We needed something which could be cloud-native architecture while on-prem," Harding explains. "We definitely knew that we needed a well-supported platform.” After assessing options, Red Hat OpenShift was selected for its ability to enable DNEG’s complex multi-cluster topology.

The new platform leverages technologies including OpenShift, Red Hat OpenShift GitOps, and Red Hat OpenShift Pipelines to automate containerization, building, testing, and software delivery. As Ian Abbott, software architect at DNEG, outlined, “What this allowed me to do was to construct, from initial git commit to final deployment, an automated system that required very little intervention from developers. The developers could just change their code and that would then automatically kick off an OpenShift Pipeline of custom Tekton tasks.

“This would then build the software, and run some tests. At the end of this, if tests pass, it creates a manual gate via a pull request on the production configuration in Git to update the release version. The developers are emailed to take a look at the test system running, and if all is good, they can just merge the PR. OpenShift GitOps, which is a Red-Hat-packaged version of Argo CD, detects that configuration change and then automatically deploys the new software images directly to all of our global clusters."

Rewards of DNEG’s cloud-native journey

DNEG’s switch to cloud-native architecture with Red Hat and automated delivery pipelines in line with continuous integration practices has yielded massive improvements. It slashed its software deployment time from one week or more to under 10 minutes per release. There has also been much greater infrastructure flexibility and ability to support innovation, and significantly increased developer productivity and feature velocity.

As Harding summarized, “Our change in architecture and the development of the delivery pipeline using OpenShift means that that pipeline from developer commits to it being in production in every facility around the world is under 10 minutes."

By removing infrastructure bottlenecks, DNEG’s VFX artists and other users now experience much faster access to the latest tools, assets, and updates needed for their cutting-edge projects.

Challenges to adopting cloud-native architecture

Implementing this sweeping modernization did involve some expected hurdles. The shift to cloud native and GitOps methodologies represented significant learning curves. Supply chain issues amid the pandemic also drew out timelines.

However, DNEG made use of Red Hat's training and expertise to skill up. Abbott credited blogs and shared best practices with guiding the configuration of optimal pipelines. Both emphasized the close vendor partnership as instrumental in designing, troubleshooting, and ultimately succeeding with the new infrastructure.

For Harding, one of the most important lessons has been the value of adopting mainstream best practices even when they don’t seem immediately relevant: “It really pays to take mainstream techniques in their complete sense … the complete mainstream definition of GitOps is the best way of doing it. 

If you lop little bits off the side ... you would have a much less resilient architecture."

He also emphasized never believing your company is too unique to benefit from standard techniques: "Never ever believe that your company or your technology is too special to benefit from those mainstream techniques. It’s just a case of how you frame your understanding."

Abbott concisely summarized his advice as: "As much as possible try to follow best practices all the way, as you're less likely to tie yourself in knots later on, and it means that you can effectively scale.”

This project is only the beginning for DNEG. With a solid cloud-native foundation now in place, Harding hinted at plans to further enhance and scale developer productivity: "We’ve got quite clear expectations of where we will be going in terms of application programming interface (API) management and developer experience – investing both in implementation of things that we know we need to do and investigation of areas that we know that we need to address.”

For an organization entrenched in a monolithic legacy, DNEG’s journey showcases the massive benefits attainable – and highlights why cloud-native architecture has become the new norm for modern application infrastructure.

This article has been updated to clarify the quotes included.