We’re currently in the midst of a technical revolution for creators...and that's you (at least for the purpose of this article). You're a brilliant storyteller, but you don't consider yourself very technically-inclined, and you aren't a professional animator.

Right now, you, without a big budget or technical skills, have little hope of making a cinematic animated film. We're here to talk about how that will change in the coming years. Sound crazy?  We’re in the middle of the revolution, not past it, and sometimes the scenery is hard to notice from the window of a car speeding by.

The Intelligent Film Tools Revolution

Filmmaking history is wed to technological advances; it all starts with someone who has a story they need to tell. And they either use the technologies that exist, or sometimes by necessity (or through sheer creativity), they invent a new method. 

When that invention is groundbreaking enough, it causes a revolution in the way movies/videos are made, distributed, and even viewed. The current era is one of those times.

Let’s list the big, important film technology revolutions. This obviously isn’t exhaustive; these are seismic shifts so big that they come to mind without a Wikipedia dive. They make you think, “movies were totally different after that.”

1.    Sound.

2.    Color.

3.    Video (camcorder) and then digital video, which made live action production easily done by all.

4.    Computer generated imagery, which revolutionized VFX, but eventually most animated movies (Pixar pioneered these).

5.    ….

What’s number five?  That one is a revolution in-progress. I’m calling it the Intelligent Film Tools revolution, because the tools themselves are smart enough to assist you, to fill in the blanks, and when designed well, get out of your way.

It’s a revolution that lets any non-technical person make an animated movie (and in time, a movie that appears to be live-action). 

How will it be possible?

You and I are lucky to be alive in an era when a suite of new technologies are maturing, and they are all necessary for you, a non-technical creator, to tell your story.

Let's survey them, from a very high level:

Real-Time Rendering and You, the Lone Creator.

Real-time rendering happens as you watch it, at the target frame rate (each frame is rendered as fast as it needs to be shown, hence the name real-time). This is opposed to offline rendering, which is the method used to make frames for most of the animated movies you’ve seen. In these cases, frames can take many minutes to render.

Unreal 5's fantastic demo is an example of real-time rendering:

Real-time rendering is meaningful to this revolution, because it changes the workflow. A storyteller doesn’t need to pre-plan every shot perfectly. Rather, they can experiment with elements like cameras, lighting, and animation, as if they were working on a stage with actors. This allows the process to be driven by a single creator, because it’s not dependent on the longer pipelines of offline rendering.

For example, if you don’t like a particular shot, you can move props, change lighting, and view the result, all yourself, without a lot of time invested, and without having to ask anyone else for help.

You have probably seen the articles on the Volume technology for the Mandolorean show. At first glance, this is just a replacement to green screen. But, its actually not possible without real-time rendering. Because, the camera is tracked in space, and the background must be drawn, every frame, to match the position of the camera:

Now vs the Future: Every year, real-time rendering improves. It may sound crazy now, but real-time technology may be able to render a photorealistic person at a cinematic level very soon. For you, the creator, this means that the line will blur between animated and "live-action" movies.

Machine Learning and You, the non-technical storyteller

If you’ve read the news, you probably see stories about deep fakes and face-swaps. These are just the tip of the iceberg for ML-enabled animation.

In the next couple of years, there will be an avalanche of useful technologies for capturing human performances, inferring animations based on voice performance, and fixing imperfect animation.

A piece of research that I loved in last year’s SIGGRAPH, was one by the Max Planck Institute, that used a single camera to capture human performances, even with baggy clothes. One of the most interesting parts of this presentation is when they chart the evolution of this technology, and how they built on the works of others. It has come a long way in five years, so we can imagine, with the same trajectory, where this might go in another five.

I've been doing motion captures off and on for about 20 years, and it continues to get easier. For it to be really useful to everyone, I have these easy guideposts:

-performance can be captured with any regular camera, in bad lighting (aka, if the machine can see you, it should work).

-your performance is captured without any technical know-how from you (after all, you're not technical). It just works.

-Your vocal performance influences the way the system constructs your animation (Based on intent of emotion).

-The resulting animation is clean, smooth, and targeted to the character, without any manual work. It never breaks based on characters with weird proportions.

‍

What that means for you? You don't need to know how it all works. You just focus on the performance, and the capture just works.

Now vs the Future: A few of the building blocks of this technology exist, many as research.  And of course, machine learning itself is improving constantly, with human pose estimation tech being pushed forward by giants like Apple, Facebook, and Google.

5G and You, the Person who'd Rather Create on a Phone

5G networks are last on this overview, because they will act as the binding agent between all of the other technologies.

We mention real-time rendering, and its potential to render images that are on par with cinematic movies. In the near future, this is only possible on a high-end computer. However, a lot of modern creators use mobile devices and laptops as their preferred tools. 5g promises to be fast enough that even a small phone will be able to remotely harness the power of a fast PC with a power-hungry graphics card, by displaying what that PC is rendering in real-time.

And in the realm of ML, some of the algorithms mentioned above are very performance and power intensive. With super fast networks, the creator is enabled with the power of whatever machine is appropriate for the algorithm, and mobile computing limitations are not a concern in their design.

What that means for you? You can focus on creation, you won’t be delayed by offline processing, and you’ll get super high quality, cinematic results. And you can use your favorite devices. Work where you want!

Now vs the Future: Right now, 5G isn’t widely available. The practical usage of these types of networks depends on the network itself, back-end technology built for this purpose, and of course, software that supports it.

The Next Pixar

Back to that idea.

The result of the Intelligent Film Tools revolution will be that storytellers, regardless of their background and technical education, will be able to tell cinematic stories.

The next Pixar? Pixar is really defined by their stories. So what everyone really wants to make is not the next Pixar, but their own stories that are seen by as many people as Toy Story.

So, the next "Pixar" could be a YouTuber of TikTok creator who designs a cast of original characters for their channel, which eventually make their way to streaming TV shows, and ultimately feature films and toy deals. This is only possible with technologies that let this talented person make cinematic-quality content from the beginning; without a budget, and without a lot of technical help.

Even more importantly, it will empower people of more diverse educational backgrounds and economic means to make animated movies, in the way handheld video cameras and phones let any talented person make videos.

The Future

Right now, this all can sound far-fetched, though I'd argue that anything innovative should sound crazy when it's in the early stages.

Let's remember that, before the camcorder was released in the 1970s, the only way to make a live action movie was to get a reel-to-reel film camera. They were mostly in the hands of the “hobbyist to professional” creators. Once VHS camcorders came out, every parent was filming their family vacations. But also, really talented people were able to make movies without a whole lot of knowledge of how to use a film camera, or the budget to buy, edit and develop film. This yielded an explosion of creativity in the indie film world.

We’re in the middle of an era that is analogous to that. High end, computer generated animated content is expensive to make, and it takes a long time. The Intelligent Film Tool revolution is based on technology that is still maturing. But when it’s fully blossomed, it will allow you, a single, talented individual to make a cinematic, high-end animated movie.  And that is revolutionary. Millions of storytellers, who otherwise wouldn’t have the resources or training it takes to make a cinematic animated film, can now tell their stories. Like revolutions before it, it is one that enables totally new voices to tell their stories, including you. And that is the most exciting part.