High Dynamic Range: Merging a Bracket, Then Checking It Against a Meter

One exposure cannot hold this scene: at any shutter speed that shows the shadowed chart, the bulb has been clipped for ten stops. Eighteen exposures can, because each frame is proportional to light and divides out by its own exposure time. The result is one radiance map, and because someone stood in that room with a colorimeter, it is a result that can be marked: a single scale factor fitted on one patch puts the other 47 within a median of 0.198 stops of the meter, and the twelve neutral patches within 0.109.

Where we are

The last lesson of image sensing. Lesson 1 said a raw count is proportional to light times exposure, lesson 2 measured what the count is made of, and lesson 3 showed why none of this works on the file a camera hands you. This lesson uses all three.

Why one frame is not enough

The scene spans 17.4 stops between the darkest metered patch, at 0.015 cd/m², and the brightest, at 2530 cd/m² [3]. Lesson 2 measured the sensor’s own range at about 11 stops. The gap is six stops, and no setting closes it.

The merge, in one line

Each frame gives a count $Z_i$ taken over a time $t_i$. Because raw is linear, the light that produced it is $Z_i / t_i$ up to a constant. Frames disagree only about which parts of the scene they can see: near the ceiling the count is clipped, near the floor it is mostly noise. Weight each frame by how much you believe it and average:

with $w$ a triangle that peaks in the middle of the usable range and falls to zero at both ends.

That is Debevec and Malik’s estimator [1] with the hard part removed. Their method recovers the response curve first, because they were working from photographs whose curve was unknown; with raw files there is no curve to recover, which is the practical payoff of lesson 3. OpenCV ships both halves, the merge and the calibration that goes in front of it when the input is not raw [5].

import numpy as np

weight_sum = np.zeros_like(planes[0], dtype=np.float64)
value_sum = np.zeros_like(weight_sum)
middle = (floor_dn + saturation) / 2

for plane, shutter in zip(planes, shutters):        # one colour channel per plane
    usable = (plane >= floor_dn) & (plane < saturation)
    weight = np.where(usable, 1.0 - np.abs(plane - middle) / (middle - floor_dn), 0.0)
    weight_sum += weight
    value_sum += weight * (plane / shutter)

radiance = np.divide(value_sum, weight_sum, out=np.zeros_like(value_sum), where=weight_sum > 0)

// OpenCV ships the same estimator, and its calibration step for the case where the
// response curve is unknown. On raw input the calibration is unnecessary.
std::vector<cv::Mat> images;         // the exposures
std::vector<float> times;            // their shutter speeds

cv::Mat radiance;
auto merge = cv::createMergeDebevec();
merge->process(images, radiance, times);   // add a CalibrateDebevec response for JPEGs

Giving it a unit

The merge produces relative radiance: correct ratios, arbitrary scale. A camera measures how much light arrived compared with the light next to it, and no photograph knows what a candela is. Turning ratios into cd/m² takes one number from an instrument.

Fairchild metered 54 points in the room with a Konica Minolta CS-100 at a 1° angle [3]. We fit one scale factor, on one patch — the lit chart’s white, at 382 cd/m². Every other point is then a prediction, and can be marked as one. This is the same shape as the survey’s own calibration, which normalises its published radiance maps with a single factor per scene.

Where the merge is wrong, and why

Every error above 1 stop is on the shadowed chart’s darkest patches, and all of them run in the same direction: the merge says there is more light than the meter did. Three causes, in the order they matter here.

Flare. There is a bare bulb in the frame. Light scatters inside the lens and lands everywhere, including on the patches that should be nearly black. The meter, pointed at a patch from close range through a 1° aperture, does not see that scatter. This is the largest effect at the dark end and it is a property of the optics, not of the sensor or the merge.

The noise floor. At 0.015 cd/m², even 30 s of exposure leaves the patch a few hundred counts above black, where lesson 2’s noise is a large fraction of the signal. Averaging over the patch reduces the spread but not the bias from clipping negative excursions at zero.

Patch geometry. The colorimeter measured a 1° spot; we average a rectangle inside the same patch. On a uniform patch that is the same quantity, and on a patch that is partly in the lamp’s shadow it is not.

None of the three is a reason to distrust the bright end, and all three are reasons to report error by luminance decade rather than as one number.

Now you try

Take the eighteen exposures down to three and watch the shadow points fall off the bottom while the median error stays quiet. Then widen the spacing: the same number of frames covers more range and leaves holes in the middle.

What the phone in your pocket does instead

Bracketing assumes the scene holds still. Hand-held, it does not, and the standard answer since Hasinoff and colleagues at Google is to shoot a burst at one short exposure and merge the frames for noise instead of for range [2]. Averaging $N$ frames divides the noise by $\sqrt{N}$, which lifts the shadows out of the floor without ever risking a blown highlight, and alignment is a smaller problem than deghosting a bracket. The dynamic range comes from underexposing and then recovering the shadows, not from a long frame.

The arithmetic in this lesson is the same in both cases. What changes is which end of the range the extra frames are spent on.

In the wild

The bench scene needed eighteen exposures. Most photographs need one, and it is worth knowing which is which before reaching for a bracket.

Twilight over La Silla, in a single frame:

What it means: this scene did not need HDR. A photographer metered it, and the entire range from the lit dome to the darkest sky fits inside eight bits with 31 clipped pixels and sixty code values still spare. Bracketing it would have bought nothing.

That is the honest counterweight to this lesson. The Luxo scene has a lit chart and a shadowed one nine stops apart, plus a bare bulb, and no single exposure holds it. A twilight landscape, for all its apparent drama, is a low-contrast subject: the sky is uniformly lit and the ground is uniformly dark. The question is never how dramatic a scene looks, it is how many stops separate the brightest thing you care about from the darkest — and the way to answer it is to take one frame and count what clipped.

Where this breaks

• A tone map is not a measurement. The pictures in this post are choices about how to show a radiance map on a screen; only the map has units. Reporting an HDR result as a tone-mapped image hides everything this lesson measured, and the literature on doing it well is a book in itself [4].
• One scale factor is doing real work. The check is of the shape of the reconstruction, not its absolute level: the level was given by the meter. Without an instrument, a merged bracket is still only relative light.
• The scene is static and on a tripod. No ghosting, no alignment, no moving subject. Real bracketing spends most of its difficulty there, and none of it appears here.
• The comparison is against the survey’s own colour matrix, fitted by Fairchild on this scene’s lit chart [3]. Errors in that matrix show up as errors in our luminance, especially on saturated patches — which is why the neutral-patch figure (0.109 stops) is quoted separately from the all-patch one (0.198).
• Green alone gives 0.264 stops. Using a single channel instead of a luminance-weighted combination costs a third of a stop of median accuracy, and much more on saturated colours. That number is in the artifact.

Where the unit lands

That is image sensing. A count is electrons times a gain, its noise is mostly the light itself, the file you open has had a curve applied, and a stack of raw frames can be turned back into light that agrees with an instrument to about a tenth of a stop.

The next thing to do with an array of numbers is process it: 2D convolution is where that starts, and edge detection is the first thing it builds.

References

[1] Debevec, P. E., & Malik, J. (1997). Recovering high dynamic range radiance maps from photographs. SIGGRAPH ‘97, pp. 369–378. doi:10.1145/258734.258884

[2] Hasinoff, S. W., Sharlet, D., Geiss, R., Adams, A., Barron, J. T., Kainz, F., Chen, J., & Levoy, M. (2016). Burst photography for high dynamic range and low-light imaging on mobile cameras. ACM Transactions on Graphics, 35(6), 1–12. doi:10.1145/2980179.2980254

[3] Fairchild, M. D. (2007). The HDR Photographic Survey. Proceedings of the IS&T 15th Color and Imaging Conference, pp. 233–238. doi:10.2352/CIC.2007.15.1.art00044 — the scene, its 54 metered points, and the D2x characterisation whose luminance row is used here: markfairchild.org/HDRPS/HDRcharacterization.html. Used for research and non-commercial publication, as its terms require.

[4] Reinhard, E., Heidrich, W., Debevec, P., Pattanaik, S., Ward, G., & Myszkowski, K. (2010). High Dynamic Range Imaging: Acquisition, Display, and Image-Based Lighting (2nd ed.). Morgan Kaufmann. — the book-length treatment of capture, encoding and tone reproduction.

[5] OpenCV Documentation (5.0). Computational photography: HDR imaging — createMergeDebevec, createCalibrateDebevec and the tone-mapping operators. docs.opencv.org/5.0