Adobe’s Project Music GenAI Control Previewed – a New Generative AI Tool for Sound

Artificial Intelligence

by Omri Keren LapidotMarch 19^th, 2024

Adobe Project Music GenAI Control is a new generative AI tool for custom music, audio creation, and editing. GenAI takes a more custom, selective, and controlled AI approach. This is very similar to Adobe’s Firefly, or the recently announced LTX Studio. Project Music GenAI Control can create music, adjust music fed into it in various ways and forms, lengthen a clip, make variations, change the mood or vibe, and more. The tool, or tool set is being previewed but is not yet integrated into any working application or software.

There’s no need to re-emphasize the importance of music, audio, and sound to the cinematic creation. It can be a complete feature score or a background of toned-down music manipulating the vibe of an interview. Sound will set the pace of a commercial video or the suspense in either a wildlife documentary or a horror film. In recent years we’ve witnessed various advancements in this field, but Adobe’s recent Project Music GenAI Control has some interesting tricks up its sleeve.

Main features

All features are theoretical at this stage of technical demonstration, but even this early in the process Project Music GenAI Control can pull up some impressive capabilities. One tool can take an input clip and enhance it in various ways. A simple text prompt will add an “inspiring film” vibe to it. Additional instruments will now accompany the initial feed, broadening the musical impression. If you’re into country music, Hip Hop, or R&B, Project Music GenAI Control will happily transform a clip into these genres. The system also provides you with control over the intensity of the newly formed tune.

Generated audio

As well as adapting and editing your original input, Project Music GenAI Control can generate complete tunes, loops, themes, and so on. Just type your prompt and you have your royalty-free clip. At that point, you can edit it as much as you want. Project Music GenAI Control can also add a portion of generated music to an existing piece. We’ve all gotten to this point where the music is a bit shorter than the video it supports. This solution, if implemented right, will solve issues like this in no time.

Real-world applications

While only previewed, it’s pretty easy to imagine the effect of such tools on the film and video industries. While complete movie scores will probably stay out of Adobe’s Project Music GenAI’s reach, at least for the near future, its effect will ripple through the industry. In my eyes, the impact will mostly affect editing efficiency across the board. It will let editors manipulate the pace, feel, and vibe of their creation with the press of a button. We’ll be able to create different variants of every video, and it’s going to be fast enough to be used as a mundane testing workflow. Background audio will become as easy as typing a sentence, with no royalties or credits required. We’ll be able to fine-tune a single tune to various audiences just by typing a text prompt. Intriguing times indeed.

Ethics and credentials

Unlike some other players in the field of generative AI, Adobe takes extra care regarding the licensing, credentials, and ethics of their products. The company is among the founding members of the Coalition for Content Provenance and Authenticity (C2PA), uniting software giants, key broadcasters, and camera manufacturers to create sustainable authentication protocols ensuring a level of trust in visual information.

Adobe is committed to ensuring our technology is developed in line with our AI ethics principles of accountability, responsibility, and transparency. All content generated with Firefly automatically includes Content Credentials – which are “nutrition labels” for digital content that remain associated with content wherever it is used, published or stored.

Adobe

Is it the next generation of AI?

AI generators have come a long way in the last couple of years, and things seem to have accelerated recently. One recent shift is towards specific control over the final product. Adobe Firefly is probably the most prominent example of this concept. The ability to transform specific selections in an image proves invaluable for many creators, including yours truly. Lightricks’ recent LTX studio poses another example and it seems Music GenAI follows that path regarding audio. Such a shift is extremely influential regarding the implementation of AI-based tools into a professional workflow. It is also a key component in democratizing more and more segments of the creative process, but not without significant pitfalls for other professionals, such as musicians who earn their livelihood off licensed music, etc.

Do you see yourself using such a tool in your day-to-day work? Will it create new opportunities for you? Let us know in the comments.