Handy AI Tools in Postproduction – Simple Ways to Accelerate Your Workflow

Artificial Intelligence

by Mascha DeikovaMay 16^th, 2023

Years ago, my colleagues and I used to joke that transcribing hours of interviews, especially in a foreign language, would be a perfect punishment for the residents in hell. Today, these tasks can be done with one click. Not perfectly, but still 100 times quicker thanks to artificial intelligence. And that’s not the only trick that this state-of-the-art technology is capable of. In this article, we focus on several AI tools in postproduction—small and practical functions that can easily take over tedious tasks and significantly speed up your workflow.

You may be currently using certain tools without realizing they rely on neural networks and machine learning models. If you’re not, you should be! Can you remember the moment when you first discovered shortcuts you could apply to your editing process? Using these new technologies can provide a similar sense of relief.

Transcription AI tools in postproduction

Since we’re on the subject, let’s begin with the generation of transcripts. This is mainly required for producing captions and subtitles for our finished films. If you edit in Adobe Space, you can automatically transcribe interviews and voice-overs directly in Premiere Pro. Just go to the “Text” window -> select the “Transcript” tab -> “Create transcription” -> apply needed “options” -> click “Transcribe”. Then simply wait a few minutes and enjoy the result. The tool can also create subs and place them according to the time code. That said, don’t forget to proofread your results: AI’s language understanding has its flaws. Here is the detailed video instruction from Adobe with all settings and options explained:

But what if we could take it a step further and apply this technology to text-based video editing? Have you ever wished for something with this capability while directing or editing a documentary? This is no longer a dream. DaVinci Resolve has added exactly this feature to their newest release of the 18.5 Beta Studio version. And following their example, Adobe did the same. With just a few clicks, you now have the power to generate a transcript effortlessly. Simply highlight the desired text portions and conveniently drag and drop them onto the video timeline. This allows you to fine-tune your rough cut precisely according to your preferences.

If you use other editing software, don’t worry, you can still enjoy the power of AI. There are so many applications on the market that it’s easy to get lost. One of the leading tools out there for creating captions and text-based editing is Simon Says AI. However, their subscription plans are quite pricy, so if you only need their services once in a while, you can pay as you go, with the price dependent on the duration of your video clip.

Some other examples of AI-based transcription are Google’s Fireflies AI, which can take written notes during online calls, or Scribie with its browser-based editor and promised 100% accuracy (Spotify and Cambridge University are among their reported clients).

Remixing music for your films in one click

Assuming your daily bread is editing corporate films or short commercials, the following AI tool might become one of your darlings. Especially if you’re familiar with the classic statement: “That’s perfect. Now, we also need a 1-minute version and three 15-second snippets.”

Adobe Remix tool, updated last fall and now integrated directly into Premiere Pro, is a genuine time and stress-saver. Powered by Adobe’s AI called Sensei, it takes your chosen piece of music and finds the ideal cut points or loops, re-timing and rearranging the song within seconds. The only steps you need to take are:

• Choose the music clip.
• Click and hold in the Ripple Edit tool group in the Toolbar panel. Select the Remix tool, then click and drag the right edge of your music clip in the timeline.
• Edit Remix properties in the Essential Sound panel. For example, set target duration, or use the parameter “variations” to quickly preview alternative remixes, focusing on the harmonic or melodic elements of the song.

Do you see these zig-zag lines? That’s where Adobe’s AI set the cuts and music transitions. Bear in mind that Remix always keeps the beginning and end of your clip, so if you want your soundtrack to start at a particular point, split the original clip and remix only the part you need.

There are various other useful AI tools specifically for audio postproduction, and we will cover them in further detail, but this one is rather practical for editors or one-man-crew filmmakers, who are tired of trying to fit a piece of music to the length of a video.

AI tools in postproduction: the magic mask in DaVinci Resolve

If you’re a seasoned DaVinci user, you probably know about this truly amazing tool, but if you’re just starting out, take a look. I’m talking about the Magic Mask. It got an additional update last year, which now lets us mask and track different objects in addition to individuals and body parts. So, how does it work?

It’s actually very simple and intuitive. Go to the color section, select “magic mask area”, and choose between masking an object or a person. Then apply it to one of the frames—preferably where your object is as visible as possible. The DaVinci Neural Engine can isolate animals, vehicles, people, and food, plus countless other elements for advanced secondary grading and effects application.

In the example above, I used this tool to quickly extract the sunglasses and convert the rest of the shot to black & white. With the magic mask, you can clone your object, put a fancy title animation behind it, or alter the color. It may need a bit of help in the selection process, but all in all, it does feel like magic. Please note that this function is only available in the paid Studio version.

Upscaling videos and pictures

Let’s look at another example. You are working with archive footage (either photos or videos) and have to drag some poorly shot random clips onto the timeline. These shots are now all fixable with artificial intelligence. If you are a DaVinci Resolve Studio user, perfect – they’ve just updated their “Super Scale” feature and added a new 2x enhanced mode (you can read all about their latest release and the AI tools in our review). If not, here are some other deep-learning applications for image enhancement you can take a look at. They can totally become a game-changer for your workflow.

One of the most widely known is Topaz Labs. It works for both videos and images and it not only increases the resolution, but it also sharpens your footage and removes the noise. However, you may also find a decent AI upscale tool for free. For example, fast Google research brought me to ImgLarger.

Some probably think that there is barely a difference in this above example, but just the tripling of image size and the ability to save the quality at the same time is still an impressive result, isn’t it?

Color grading with AI

For colorists and color scientists, we have a separate review that explains how to automate color management, easily match shots from different cameras, as well as develop beautiful show looks using AI. Head over and you might be surprised to find that even the very popular large language model ChatGPT can do wonders for your color grading.

All of this is only a fraction of the available AI tools in postproduction that can speed up your workflow in a few easy steps. So, let’s turn the tables now. Which artificial intelligence technology and machine-learning-based functions do you use and can recommend to the community? Let’s create a list of handy tools in the comments below!

Featured image: screenshots from Simon Says AI, DaVinci Resolve, Adobe Premiere Pro, and ImgLarger.