CamTrackAR and iPhone 13 – Mobile Virtual Production and PreViz Toolkit

Software News

by Yaroslav AltuninSeptember 30^th, 2021

Smartphones are slowly transcending their moniker as new phones are released with bigger and better tech. Recently the new iPhone 13 came out with faux depth of field for video and ProRes, a professional video codec soon to be added in an update. But as the hardware develops, so does the software. With it, users are able to create amazing things and find unique ways to supplement their cinema gear. CamTrackAR is one such piece of software. The CamTrackAR and iPhone 13 combo are able to give users a unique way to approach virtual productions and previz for VFX-heavy projects.

CamTrackAR is developed by FXhome, the creators of HitFilm, a video editing, and VFX software package. This multi-talented crew of coders and creatives brings powerful tools and education to filmmakers and artists. It doesn’t matter if you’re just starting out or hitting yet another milestone. We recently published an article about the release of CamTrackAR 2.0, which added some very important features. With the updated CamTrackAR and iPhone 13 combo, FXhome aims to bring Hollywood-caliber virtual production technology right to your fingertips. Between pre-visualization, in-phone chroma key, and integration with Blender, Unreal Engine, and their own HitFilm, CamTrackAR looks to be a diamond in the rough. But with great power, also come some limitations.

The great power

Touted as the world’s first mobile virtual production studio for iOS, CamTrackAR imbues your iPhone or iPad with extraordinary gifts. By using Apple’s ARkit and LiDAR scanner technology, FXhome produces accurate 3D camera tracking data while simultaneously capturing video. iPhone users can then key out green or blue screens, and import 3D virtual backgrounds and dynamically place them in the environment. The floor, wall, and background planes can then be set with a tracking grid and anchor points. All at the press of a button. The CamTrackAR and iPhone 13 combo will then record and do all the work for you. A shoutout to Film Riot for their awesome video on the tech:

Once the camera tracking data and footage have been gathered, it can all be exported to HitFilm, Blender, AfterEffects, Apple Motion, and AutoDesk programs. FXhome also offers .json and .csv formats, allowing users to process the data in their own custom-made program. While some of these features are free, advanced features like unlimited export, multiple point anchors, and 3D model import are only available with a monthly subscription or outright purchase. Check out the FXhome help guide for a detailed breakdown of all the features.

Some limitations

While it seems like a perfect setup, there are limitations to worry about with CamTrackAR. As of this post, the 3D background elements can’t be burned into the video. While you can see the key on your phone, it won’t record that way. Theoretically, you could get around this by recording the iPhone screen with the iOS screen record feature. Unfortunately, we haven’t tested this. On top of that, the workflow to get 3D images into your phone is currently a bit cumbersome and requires some beta software to use. Thirdly, recording options are limited to 30fps and 60fps and won’t utilize the camera stabilization inside the iPhone. According to FXhome, these are limitations of the iPhone’s ARkit.

Do you need it?

Will it film the next Avatar movie? Probably not. However, this app is a dirt-cheap option to help expand your VFX tool kit. It’s free to download and has a huge list of usable features. The upgrade is only $29.99 and for that price, it’s worth the cost of entry. At the very least, it’ll be an amazing educational tool. At best, it can expand your workflow, increasing your efficiency and creative output. This saves money and time. Which are both critical to any production. Visit FXhome for more info on CamTrackAR.

What do you think of this app? Is this something you want to see in your tool kit? Are there features you wish it had? Let us know in the comments.