How Canon Measured 15 Stops of Dynamic Range on the C300 Mark II

ReviewsCameras

by Sebastian WöberNovember 30^th, 2015

In September we published our lab results of the Canon C300 Mark II dynamic range. Most interesting about this test was the fact that according to our own parameters the dynamic range was closer to 12 stops, while Canon markets 15-stops of dynamic range for their current flagship cinema camera. This caused some confusion among customers and earlier this month Canon replied to our article and offered an insight into how they measured dynamic range.

Dynamic Range is one aspect of a camera’s performance. Certainly there is more to a camera as a whole than just this one figure. Nevertheless, for many cinematographers a high dynamic range is an important attribute of a camera sensor.

About measuring dynamic range in general it is important to mention that at this moment there is no established “standard” to conduct such a measurement. This is why companies come up with their own standards and are able to market their cameras with very high dynamic range values, which of course is confusing to the customer. Here’s where we decided to measure all cameras with an equal and realistic standard in order to be able to compare them to each other.

First let’s look at how we measure dynamic range at cinema5D and later we will describe Canon’s testing methodology as presented to us.

How cinema5D measures dynamic range

As some of you know accurately testing dynamic range is a somewhat delicate task as each sensor has a very unique way it captures light. In order to assess how many of the stops of data the sensor outputs are “usable” for a normal production process, every tester has to come up with testing parameters they can apply to the cameras tested in order to compare them to each other. At cinema5D we strived to setup these parameters in a way, so that we get a realistic evaluation of each camera’s performance.

We test dynamic range of a camera at its optimal settings and film a DSC labs XYLA-21 transmissive test chart with a Zeiss 50mm CP2 T/2.1 makro lens. After that we use IMATEST evaluation software to measure the shots objectively. The aforementioned parameters are applied by using a fixed signal-to-noise threshold for all evaluations.

In our tests we rated the Canon C300 mark II at 12.3 stops, about 2 stops weaker than the Arri ALEXA that is rated by us and by the manufacturer at about 14 stops. We also pointed out the horizontal strip that appears on overexposed portions of images recorded with the Canon C300 mark II.

For the image above the gamma has been raised to an extreme manually and is only for you to observe how the two mentioned cameras record light differently and where the noise levels surpass the information recorded. This version of the shot is not used by us to determine the dynamic range.

At the point where the noise is too strong in the dark areas of your image, it becomes hard to use that data for your film. When evaluating dynamic range every tester has to determine such a point, where the noise is too strong to effectively use the underlying data.

How Canon measured dynamic range

We’ve had an insightful conversation via e-mail with Senior Fellow of the Professional Engineering & Solutions Division at Canon, Larry Thorpe, who has also written Canon’s whitepapers on the Canon C300 mark II, which is an in-depth look at the technology behind the new camera.

In a document that Canon officially sent out to the marketplace on the topic of dynamic range Canon writes:

“Canon recognizes that some of our customers are commenting about the image quality results after shooting the Canon C300 Mark II with the new Canon Log 2 Gamma setting.

Canon Log 2 Gamma is designed to provide as much usable data and dynamic range as possible for professionals to make use of in post-production. Because of this, it is possible under certain shooting environments that there can be the appearance of increased noise in low-light areas and/or luminance banding. These two artifacts are intrinsic to the design of the CMOS sensor. To make a fair assessment of Canon Log 2 Gamma the image should not be judged by viewing the log output alone, but rather the final graded image. (…)”

In order to reduce the noise mentioned above Canon further recommends users should:

• “Set camera ISO to 400.”
• “Set the cameras internal noise reduction filter to 2.”
• “Utilize the original Canon Log Gamma (To minimize Luminance Banding).”

Canon later says that the base sensitivity for Canon Log is ISO 400, while the base sensitivity for Canon Log 2 is ISO 800.
About their testing methodology Canon goes on to say:

“Measurement of Dynamic Range can be both subjective and objective and Canon realizes there can be a variance in this determination based on procedures used and the tolerances of the end user. Because of this we are presenting our methodology of testing and determination of dynamic range visually.”

For their tests Canon used the Canon EF 50mm F1.8 lens at F/5.6 with a shutter of 1/68th and the aforementioned chart. It turns out that Canon used the same DSC labs XYLA-21 transmissive chart that we at cinema5D are using for our evaluations. In theory the testing methods seem to be very similar to ours, with the only difference that we used the Zeiss 50mm CP2 T/2.1 makro lens and a PAL framerate with a 1/50th shutter and F/4.0,5.

The obvious difference though, is that Canon evaluates the dynamic range with a Waveform Monitor (Leader L5770 Multi monitor) and a subjective readout while cinema5D is using the IMATEST to evaluate the results objectively.

Canon sent us a screenshot of the results on their waveform monitor. Each step represents a step on the chart. The first step represents overexposure, the second step  represents the first stop of dynamic range, which Canon also explains in their document.

Larry Thorpe further commented on the signal-to-noise ratio they determined as the threshold we described as “usable dynamic range”:

“(…) on a Millivolt video amplitude scale (with Reference White at the traditional 700 mV) the linear signal output (…) – for those last two stops – are less than one millivolt in amplitude. Even with the impressive 67dB Luma signal to noise specification this means those two steps ARE noisy. Indeed, the definition of Dynamic Range is when the final lower step has a signal to noise of approximately 0 dB.”

In summary it looks like the testing methods of Canon and cinema5D are very similar if not mostly identical, with the only difference of where to apply the threshold in order to determine how far into the noise the dynamic range is counted. cinema5D’s evaluation software did not count the 13th stop, sitting inside the noise floor, as another stop of dynamic range, while Canon did, as they did for stop 14 and 15.

Conclusion

First of all we are happy that Canon decided to put their testing methods on determining the 15-stops of dynamic range for the Canon C300 mark II up for discussion. This shows us that Canon takes user feedback and concerns seriously and it supports the notion that the Canon C300 mark II is an important camera to them. Furthermore we felt a lot of detail and thought went into the scientific evaluations they presented so there is no question they understand the science behind sensor design and are willing to share details about their technology.

Finally, we want to leave the conclusion about the testing methods presented up to you. Is it simply a different way to evaluate dynamic range? Was Canon’s evaluation parameter of 0db as a threshold too loose or are cinema5D’s testing parameters maybe too strict?

My personal note: These tests are useful to us cameramen when there is a reference we can compare the results to. This is the mission of cinema5D, to give users an impartial comparison between cameras, free from any marketing agenda, so they can make an informed decision based on the performance needs of their shoot.