8K and 4K HDR Smartphone Video Recording with the Snapdragon 865

News

by Richard LackeyJanuary 31^st, 2020

Capturing 200Mp photos and recording 8K video with your phone is right around the corner thanks to Qualcom’s Snapdragon 865 chipset.

Not to be outdone by Apple, at least not for long, the newest generation chipsets to power this year’s Android devices are delivering some surprising capabilities. The Qualcom’s Snapdragon 865 chipset will power many 2020 Android smartphones and ups the ante considerably in the imaging department.

Before you all skim over everything I’ve written here to torch me in the comments, of course, 8K video recording in a smartphone is ridiculous. All kinds of technical issues and problems (the encroaching limits imposed by the laws of physics being one) immediately come to mind.

Logic, rationale, and physics be damned, it’s going to happen regardless because people will part with their hard-earned cash for something that says 8K on it, whether it allows them to watch 16x upscaled HD SDR content on an 8K HDR TV, or record the daughters ballet recital in 8K with their fancy new smartphone. However, try to keep an open mind because the tech is interesting, and it might actually end up being useful.

The heat is on smartphone manufacturers to up their photo and video capabilities now more than ever. Our smartphones have become the defacto standard camera everyone carries around, even us video professionals, still when carrying a mirrorless or DSLR, or a cine camera as well. The smartphone as a camera is ubiquitous, easy to use, socially connected, and they are now capable of meeting the needs of pretty much any and all casual photo and video use.

Ultra-high-resolution photos and videos are coming to our smartphones, but it’s important to keep the facts straight and not fall for all the hype. At the same time, there may be some real advantages on the go creators can leverage from.

The Snapdragon 865’s Next Generation Spectra 480 ISP

The megapixel race has come to smartphones. In a space where cameras are largely defined by software, there is some key hardware that makes this possible. This starts with the ISP, or image signal processor. The overall bandwidth and sheer processing power of an ISP are what determine the limits of resolution, color bit depth, and frame rates of a particular device.

The Spectra 480 ISP in the Snapdragon 865 supports up to 200Mp photo capture, 8K video capture, and 4K video capture in high bit-depth HDR. Not only can it run the required math and process all those pixels, but the Spectra 480 is also tied into Qualcom’s fifth generation A.I. engine. This allows the device to perform complex image segmentation in real time, identify objects, and backgrounds.

Larger, Higher Resolution Imagers

An ISP and chipset that supports extremely high resolution photo and video capture is nothing without an image sensor that offers up all those pixels. The more raw photosites there are, even if they are tiny and noisy, the more clever math can be done downstream to computationally extract useable image information.

As an example, the Sony IMX586 is a Quad Bayer design that offers 8000 x 6000 pixels in a 1/1.2” size sensor. The “Quad Bayer” CFA effectively groups 2×2 blocks of photosites under each color filter. This results in less chroma resolution than pixel count but is a necessary trade-off for such a high density, small size imager. The ability to cleanly improve the signal to noise ratio and chroma fidelity when you’ve got four samples to work with instead of just one can outperform a single larger photosite. At the same time, it provides more discrete samples when calculating luminance at the full pixel count.

There will no doubt be more innovations in smartphone sensor design to come this year as the battle for sales rages on. It will be consumers and the growing niche of mobile video enthusiasts that will be the winners. It usually doesn’t take long for the technology launched in premium handsets to appear in more budget-friendly phones, and in many cases, the gap in features between entry-level, mid-level and high-end phones is getting smaller.

Snapdragon 865 Camera Specifications

• Image Signal Processor: Qualcomm Spectra™ 480 image signal processor, Hardware accelerator for computer vision (CV-ISP), Dual 14-bit CV-ISPs
• Dual Camera, ZSL, 30fps: Up to 25 MP
• Single Camera, MFNR, ZSL, 30fps: Up to 64 MP
• Single Camera: Up to 200 MP
• Camera Features: HEIF photo capture
• Slow Motion Video Capture: 720p @ 960 FPS
• Video Capture Formats: Dolby Vision, HDR10, HDR10+, HEVC
• Video Capture Features: 8K video recording, 4K video capture with simultaneous 64MP photo capture, Rec. 2020 color gamut video capture, Up to 10-bit color depth video capture
• Codec Support: Dolby Vision, H.265 (HEVC), HDR10+, HLG, HDR10, H.264 (AVC), VP8, VP9

Keeping it Real

The actual resolution does not equal sensor pixel count.

It is worth mentioning that there is a big difference between the actual resolving power of any camera system, and the number of photosites an image sensor may have. Also, just because a camera records a file of certain pixel size, doesn’t mean that the image actually holds the same amount of detail.

Marketers like to hype pixel counts but it’s only one small part of a much larger equation. This is especially true of miniaturized camera modules as small as those in smartphones. As good as they are, and they have become excellent, the real ability to resolve detail in a smartphone image comes down to the design of some very small optics, the design, efficiency and performance of the sensor itself, exactly how sensor data is debayered, and of course all the math and video compression that goes on downstream.

The Advantages of Downsampling

There are some advantages to high pixel counts, even if some, or a lot of it doesn’t contain much extra information. When I first started shooting video with an iPhone, I was recording 4K at the highest bit rate possible in order to get a better looking downsampled HD image in post.

Taking the same approach to 8K smartphone video (very likely to remain 4:2:0) may produce a pretty decent downsampled 4K result, especially if manufacturers move to 10-bit encoding. Of course, we’ll have to wait and see how all this is implemented and what kind of video files 8K capable smartphones will actually give us.

The Smartphone Camera is Here to Stay

Smartphones as photo and video cameras are definitely here to stay, and they will continue to get better at it. They will become less noisy, and better in low light. Higher color bit depth is coming, AI and machine learning is helping to fill in the gaps.

What do you think of 8K video recording on a smartphone? Are you convinced this trend is headed somewhere useful or will this always be a compromised camera for consumer use only? Let us know what you think in the comments.