Camera Manufacturers: Don’t Release Unfinished Products (and Ask Us to Review Them) – A Story about GoPro and DJI

Opinion

by Gunther MachuJanuary 4^th, 2024

Fair warning – this article is a personal opinion piece pointing my finger at the dubious practice of throwing unfinished cameras onto the market. Are you curious to hear what happened to me when I was supposed to test the new GoPro Hero 12 alongside the DJI Action 4? Then read on...

It’s the end of September. As every year around this time, new versions of the popular action cameras from DJI and GoPro have already been announced, more or less ready to hit the shelves. In my particular case, both the GoPro Hero 12 and the DJI Osmo Action 4 were sent to CineD to be reviewed. As we typically try to use new cameras on “real” jobs, we discussed what we could do with the two new action cams.

GoPro Hero 12 Black and DJI Osmo Action 4 – the specs

Please have a look, as we covered the specs of both cameras in articles here and here.

Just the mere numbers/specs on both cameras didn’t do much to excite me as they were too similar to their predecessors – but there were a few things to take a closer look at, among them GoPro’s new 10bit GP-Log encoding mode that caught my interest in the official press release kit:

It is advertised (also on the GoPro website) as the thing to choose for “professional creators” for “the most lifelike colors and to max out dynamic range” along with dedicated LUTs before post-production. Wow. Finally, “pro” features are trickling down to action cameras. They have truly come a long way.

The DJI Osmo Action 4 offers a similar feature, namely 10bit D-Log M with a dedicated LUT plus a new 1 / 1.3″ sensor which should provide more headroom for lowlight shots. Great!

Planning the review of both cameras for my upcoming travels

Hence, as I was venturing on a long business trip to India, China, and Egypt, starting in early October and ending beginning of November I offered to take both cameras along and do an in-depth side-by-side video review over the four weeks.

Along the trip, I was planning to shoot a wildlife documentary about wild wolves in India and then finish everything off with a diving vacation at the Red Sea in Egypt.

I picked up both cameras from the CineD headquarters, installed the latest app & firmware versions, and headed to the airport two days later. Obviously, I had no time to run any tests with either camera but as I already owned GoPro cameras since the Hero 2 and had shot a lot with the recent DJI Action 3, I was confident in reviewing these new cameras – I mean, what could possibly go wrong?

The Indian Wildlife Documentary

To capture driving scenes in India, I decided to use only the GoPro Hero 12 with the new 10-bit LOG mode and the recent firmware 1.12, as I didn’t want to mix and later match footage from both DJI and GoPro.

The documentary turned out to be quite a difficult task as the monsoon rain had not finished yet and we got stuck multiple times off roads while searching for traces of wild wolves and other grassland animals. No risk, no fun, right?

But finally, after four days our efforts paid off and we managed to shoot wild wolves in the very early morning. It was an amazing experience. A pack of five wolves was dancing and playing right in front of us. We also interviewed local village people on their relationship with the wild animals and I am excited to finish working on this, but this is another story.

I had no time to review the GoPro-Log footage as we had to get up at 4 am every morning and usually finished our days after midnight as we also captured fantastic night creatures like chameleons.

But why worry? After all the GoPro footage looked good on the small rear display. Admittedly, if this was a paid job, I’d probably had found the time to check footage on a bigger screen right after shooting, but there was simply no time in this case.

Finishing the trip with diving at the Red Sea in Egypt

During my time in India and China, I was busy fulfilling some work duties and had a rather good feeling that I had canned everything for my wildlife documentary. Out of curiosity, I downloaded files on my business laptop while traveling but the H.265 files stuttered heavily during playback so I just saved it for later.

Then, after nearly four weeks of traveling, I was glad to end the trip with my family in Egypt. Finally, I wanted to do an in-depth comparison between the Hero 12 and Action 4. I captured a lot of footage simultaneously on both cameras and while diving, I only used the DJI Action 4 as it’s waterproof until 18 meters (Hero 12 only 10 meters).

In hindsight, I was lucky to capture a lot simultaneously. Here is why.

The unbelievable case of the GoPro HERO 12 LOG encoding mode – the unfinished camera

Now, in Egypt, I started downloading some of the HERO 12 GP-Log footage via the Quick App and exported video frames to be sent to my friends via WhatsApp. They looked rather soft, but I attributed it to the reduced resolution download, and delivery was via WhatsApp – so what?

Near the end of my holidays, the weather turned bad, underwater visibility was gone and I finally opened my son’s M1 Macbook Air to review footage (it cuts through the H.265 footage like a hot knife through butter). I fired up DaVinci Resolve and started to make some first comparisons.

Looking at the GP-Log footage I couldn’t believe my eyes. I proceeded to review all settings, started to google, and looked up videos on YouTube… and got confirmation.

The GoPro HERO 12 LOG encoding mode looked like 1280×720 footage from the early days, especially the 4K 60p mode. Have a look below:

Just look at the details in the fore- and background. For example, the divers: the one on the left is barely visible in the GoPro footage. This fact is even more obvious when comparing it to the DJI Action 4 footage, which is very nice and detailed without being oversharpened – what is going on here?

I then thought that the footage underwater in low light may be more noisy, hence the smaller GoPro sensor has a disadvantage.

I then looked at additional side-by-side shots in bright sunlight. Here is a 200% zoomed-in comparison, using the respective LUTs from both companies:

Just look at the dark side of my face – my skin tone is brownish-greenish, and there is a sudden transition from brown to green/yellow in the shadows. GP-Log footage is lacking any sort of detail, skin tones are muddy and plasticky and the whole image is completely washed out. Color banding is all over the place – this is supposed to be 10bit? Hard to believe.

Whereas DJI looks natural, detailed, and overall very good.

So. Here I was, luckily having filmed my family vacation on both cams but having used GoPro and the new GP-Log mode exclusively on my wildlife documentary – “Houston, we have a problem”. It’s the 28th of October, more than six weeks after the release of this camera, still promoting the GP-Log mode for pro-creators on their website with firmware version 1.12.

Version 1.xx usually hints at a final product firmware and not an experimental, beta-tester feedback version.

I have been a GoPro customer since the Hero 2 (10 generations ago) and have put a lot of trust in this company – up until now, they have always delivered a superior image. How is it possible they released such a product? I am seriously asking myself if anyone over at GoPro has even tested the GP-Log mode before the release!? Is this supposed to be the pro-creator mode?

End of story. Or is it?

Having swallowed my initial disappointment I used the remaining time of my holidays to finish the video review by recording myself speaking into the camera while on the beach, highlighting all the deficiencies and issues. I will publish this rather soon. Or at least, that’s what I thought.

On the last day of my holidays – the 31st of October – I hooked up the GoPro to the Quick App again, and voilà – a new firmware version 1.2 just came out. “Fixing image issues with GP-Log encoding”.

Also releasing new LUTs. And yes, definitely fixing a lot of the issues described above.

Finally – good news for the product, but it killed my review

But not so good for me. My review was 99% ready with the main message being to avoid GP-Log encoding mode at all costs. Now all I can do is dump it – hence, there will be no video review this time. I guess that’s part of the life of a camera tester.

However, reflecting on this story neither of my conclusions makes me happy. You either do a very quick test and publish your findings immediately out of fear that a new firmware comes out which makes your findings obsolete. Or you don’t do tests at all and just copy & paste news headlines and specs. A long, and in-depth review seems to have no place in such a scenario.

Last but not least a word or two on the DJI Osmo Action 4

Nevertheless, there is an apparent message from my comparisons, even with the latest fixes for the GoPro HERO 12. We finally have a new king of action cameras, the DJI Osmo Action 4. The image is very detailed without being oversharpened, in all modes. 10bit D-Log M along with the official LUT works very well, the sound quality from the microphones is surprisingly good and from day one it performed flawlessly without any issues.

Very good job on the Osmo Action 4, DJI! Congratulations to the new king of action cameras!

My appeal to camera manufacturers – don’t release unfinished cameras

Dear camera manufacturers, if you want us camera testers to be serious and pour a considerable amount of work into video reviews, we kindly ask you to focus all your efforts on finishing a product before releasing it. If you can’t finish a feature in time, then please let us know about your firmware roadmap and resist the urge to market features that are not working properly.

I am not against firmware updates. Firmware updates show the commitment of manufacturers to support their products. But this is a camera intended for serious use, not a banana that ripes at the customer’s premises – there is no reason to release a product when its software isn’t ready, because that just frustrates customers and professional reviewers like us. There’s no shame in pushing a release date in favor of releasing a more final product that actually delivers on what is advertised.

The case of the DJI Mini 3 Pro – different manufacturer, same story

My colleague Nino Leitner had a very similar experience with the DJI Mini 3 Pro when it was released in May 2022. We received the camera early for review as it often happens (under embargo before the release date), and Nino worked on an extensive video review of the drone, which was finished and ready to be published on the day of the release. Then, on that same day, DJI released a significant firmware update that we were not informed about beforehand – adding 10-bit video recording. Great news for the customer, not great news for the reviewer – the finished video review was canned and never published, just like the review article. Nino was frustrated and wanted to update the review eventually, but seeing that many other reviewers apparently did have access to a beta firmware with the 10-bit color space, he skipped it altogether, which I totally understand.

Manufacturers rely on us, the press, to make reliable camera reviews to review their products and also get the word out about them. We invest lots of sweat and time (and money for editing, running a company …) in these reviews, and deserve the respect to be sent cameras that deliver what is promised, especially when it’s cameras that already start shipping at the time of the embargo release (it’s a whole different story if we touch a pre-production camera, knowing that features are still missing that will be added later).

Dear readers, what are your experiences? Have you been let down before? Let us know in the comments below …