LTX Studio Deep Dive – The First AI-Based Full Editing Suite?

Artificial Intelligence

by Omri Keren Lapidot12 days ago

LTX Studio is one of several interesting Generative AI video applications. It represents a relatively unique concept, emphasizing continuity and selective adjustments. On top of this, LTX Studio doesn’t end up creating generated footage. It’s a full editing suite, including a timeline, sound, camera motion, lights, and effects. While their current end-product can’t rival filmed footage, it can be an excellent tool for pitching, writing, scripting, storyboarding, etc. In this article, we’ll review some of the initial semi-independent efforts in developing the software.

Before we dive in – all videos here are based on a preliminary version of LTX Studio This means many things may change by the time it hits the market, and it also means that the creators of the videos have a certain relationship with the company (Lightricks). With that in mind, Lightricks does share an insightful look into the actual workflow in ways most other AI companies refrain from. These videos provide a fascinating glimpse into what may well be the future of content creation and filmmaking- at least a growing part of it.

The first thing that grabbed my attention was the choice Sebastian Kamph made to follow the preset storylines instead of starting with an original prompt. I could attribute it to the urge of “Just let me play with this new toy!” but it seems as if this practice was also the choice of other creators, such as Tim Simmons of Theoretically Media, whom you might remember from the company’s launch event. As the app is still in its early stages, this doesn’t seem too problematic, but if it’s still limited to preset prompts I believe that should be disclosed. The company may have taken the more responsible route, limiting early testers to limit what may come out. Generative AI systems may prove unpredictable so caution is advised.

Into the storyboard

Things seem to significantly improve when diving into the workflow. After a relatively short loading time (though editing may be involved here) the system will provide a title and a set of characters based on the prompt or “Synopsis” entered earlier. The user can choose from several different aspect ratios and cinematic styles (although I’ve only seen samples of the default “Cinematic” style so far, with hints of other styles like the one previewed here). Cinematic styles are all subject to changes in later stages of the project, or if you want to try several distinct looks. In my opinion, it’s one of the more useful tools of LTX Studio, but more on that later.

Consistent dynamic characters

One of LTX Studio’s unique features is its ability to generate consistent characters throughout the entire storyline (and end product). Each character is individually editable. Every edit may be “refreshed” to generate a different AI take on the same prompt. The prompt is also subjected to changes before and along the editing path. The early version also allows for an existing image drop-in for face swap, but I must say – this feature seems rather problematic. It will allow for easy deep fakes and may introduce a plethora of legal issues. So, while potentially useful, I’m not sure we’ll see it in the final version.

Now we press “Start” and see the magic being generated

The synopsis and “cast” are set, and now LTX Studio can generate the story structure. Here we see the best evidence of the unique conceptual perception behind LTX Studio. It’s not about generating the best-looking footage, it’s about the story infrastructure. The software/app expands the synopsis, breaking it into scenes with each scene divided into specific shots.

Here you can control various aspects of each shot. Wide, close-up, medium shot, low or high angles, motion scale, and the prompt itself are all changeable. You can also switch between shots by dragging and dropping them. The left panel will affect the entire scene, deciding on location, lighting, and weather conditions (although additional shot adjustment is sometimes necessary.) There’s also a sound tab enabling voice-over and ambient sound in two different channels, each controlled separately.

Shot editor

The shot editor enables further control over each shot in a separate panel. Here you can set a specific camera motion preset, adjust its intensity, and add a negative prompt for better control of what shouldn’t be in the frame. An image prompt where you upload a reference image should become available in the future.

Duration and frame rate are also set here. The custom camera motion is pretty impressive, letting you define a relatively precise motion path in a semi-3D environment generated from the shot.

Additional features

LTX Studio is a feature-packed app. To my eyes, it’s more closely related to full-blown editing software such as Adobe Premiere, DaVinci Resolve, or Final Cut Pro than it is to other AI generators. In addition to all the features discussed above, plenty of others are available – face swap, inpainting (to paint out specific details), etc. Different video styles are also available (Watercolor, 3D model, Anime, etc.), and the company emphasizes selective adjustments and continuity over other features.

Current use cases

At this point, LTX Studio struggles to provide footage that one could mistakingly perceive as camera-originated. So, while we may not create our next feature film with it soon, it can still be useful as a scripting, pitching, or storyboarding tool. Its most impressive achievement is the proof of concept of an AI-based video editing tool. Here lies the core difference between LTX Studio and its parent company, Lightricks, and other players in this field. Lightricks is a tech company but much of its staff comes from the creative world – designers, photographers, filmmakers, etc. Therefore, its products emphasize creative workflow rather than algorithmic tour-de-force.

End result

If we are to judge LTX Studio by the cinematic quality of the end result at this point, it’s still far away from challenging current camera or CGI-based imagery. Coming from Lightricks, it’s no surprise that it mostly looks like a still image with some motion added in post. All “signature moves” of earlier AI image generators are also here, with weird fingers, awkward postures, and inauthentic dialogs. But as bleak as it may sound, I believe the most influential product LTX Studio offers is the UX design. To the best of my knowledge, this is the first attempt at creating full-fledged editing software around generative AI. It demonstrates some of the incredible potential that will probably shape our industry in the years to come.

Can you see yourself giving AI to control any aspect of your video creation from the idea to editing like LTX studio suggests? Please share with us your thoughts in the comment section below