play’s Holly Announced – Multi-Modal AI to Speed Up Video Editing

Editing SoftwareIBC 2024

by Mascha DeikovaYesterday

IBC is not only about new gear and software updates. It’s also the chance for young emerging companies to present various technical solutions for filmmakers and video creators. Thus, an Austrian company called “play” demonstrated their multimodal AI for video editors, which can create a rough cut based on your text prompt. Let’s take a closer look at play’s Holly and how it works.

According to the company‘s press release, the main goal behind play’s artificial intelligence model “Holly” is to develop a tool that will take over mundane tasks (such as adding metadata and selection process) and free up time for the creative parts of editing. At first sight, sounds like a reasonable use of AI capabilities, doesn’t it?

We’re also excited to announce, that play is one of our CineD Best-of-Show Award winners at IBC 2024. They’ve received our AI Innovation Award this year, as we see a lot of potential in their demonstrated tool.

play’s Holly: the essence

play’s Holly is a browser-based application that uses machine learning to analyze delivered video clips and provide them with comprehensive metadata. Apart from that, the software includes LLM (a large language model) to communicate with users and create a rough cut based on the text description they provide. Allegedly, Holly is also able to understand speech in video clips in 99 languages.

You save up to 80% of your time and can concentrate fully on the creative part: the perfect edit.

A quote from play’s official webpage

How does play’s Holly work?

First, you need to upload your footage into the software. Alternatively, you can access it via their direct integration with frame.io. Once you’re done, Holly analyzes the delivered video clips and creates a library of metadata that looks like this:

As we can see, play’s AI includes information on the shot type, scene description, keywords, character IDs, and even color palette of the shot. According to the developers, Holly “understands” the context and knows, for instance, which camera angles belong together.

The second step is to prompt the idea for your rough cut and refine it using a simple chatbot. In the demo, developers use the following wording: “Create a timeline focusing on dirt bikers as extreme sports highlight wheel with at least 7 clips.” Here you go:

After Holly creates the timeline with the requested rough cut, you can either refine it directly inside the tool or export it as a file to the video editor of your choice. All the metadata markers, made by AI, will also be transferred, and this alone is already a great feature for speeding up post-production.

Developers mention it’s only the first step, and in the future, they also want Holly to learn from and mimic user’s favorite editing styles.   

play is already working with some well-known production houses and videographers to co-develop the prototype quickly into a product.

Early access to the tool

Holly is still young and will need a lot more training. The team behind play told us that this is why every test user is actually also a pioneer and very important for them – as well as their feedback.

Our small, but highly experienced team is working flat out on further development. Further models are being implemented, new features such as preview are being integrated and plugins for the most important NLE editors are also on the roadmap for 2025.

If you’re interested, you can also apply for free early access by clicking on the yellow button here.

Although I can already imagine the backlash from professional editors (and assistant editors, for that matter), a lot of content creators will indeed welcome this tool as a useful solution to speed up their post-production. So what’s your take on the play’s Holly AI? Would you try it out? Can you imagine implementing it into your workflows? Let us know in the comments below!

Feature image: a collage of images from the play’s presentation.