Exposure Fusion

编程小6 (6) 2024-07-08 15:23

Hi，大家好，我是编程小6，很荣幸遇见你，我把这些年在开发过程中遇到的问题或想法写出来，今天说一说
Exposure Fusion,希望能够帮助你!!!。

Abstract

We propose a technique for fusing a bracketed exposure sequence into a high quality image, without converting to HDR first. Skipping the physically-based HDR assembly step simplifies the acquisition pipeline. This avoids camera response curve calibration and is computationally efficient. It also allows for including flash images in the sequence. Our technique blends multiple exposures, guided by simple quality measures like saturation and contrast. This is done in a multiresolution fashion to account for the brightness variation in the sequence. The resulting image quality is comparable to existing tone mapping operators.

我们提出了一种技术，将包围曝光序列融合成高质量的图像，而无需首先转换为HDR。跳过基于物理的HDR组装步骤可以简化获取管道。这避免了相机的响应曲线校准和计算效率。它还允许在序列中包含flash图像。我们的技术通过简单的质量测量，如饱和度和对比度，将多种曝光混合在一起。这是以多分辨率方式完成的，以解决序列中的亮度变化。得到的图像质量与现有的色调映射算子相当。

1.Introduction

Digital cameras have a limited dynamic range, which is lower than one encounters in the real world. In high dynamic range scenes, a picture will often turn out to be under or overexposed. A bracketed exposure sequence [5, 17, 26] allows for acquiring the full dynamic range, and can be turned into a single high dynamic range image. Upon display, the intensities need to be remapped to match the typically low dynamic range of the display device, through a process called tone mapping [26].

数码相机的动态范围有限，比人们在现实世界中遇到的情况要小。在高动态范围的场景中，照片往往会曝光不足或曝光过度。包围曝光序列[5,17,26]允许获取完整的动态范围，并可以转换成单个高动态范围图像。在显示时，需要通过一种称为色调映射的过程重新映射亮度，以匹配显示设备通常较低的动态范围[26]。

In this paper, we propose to skip the step of computing a high dynamic range image, and immediately fuse the multiple exposures into a high-quality, low dynamic range image, ready for display (like a tone-mapped picture). We call this process exposure fusion; see Fig. 1. The idea behind our approach is that we compute a perceptual quality measure for each pixel in the multi-exposure sequence, which encodes desirable qualities, like saturation and contrast. Guided by our quality measures, we select the “good” pixels from the sequence and combine them into the final result.

在本文中，我们建议跳过计算高动态范围图像的步骤，立即将多次曝光融合成高质量、低动态范围的图像，准备显示。我们将此过程称为曝光融合; 我们的方法背后的想法是，我们为多重曝光序列中的每个像素计算感知质量测量，该测量编码所需的质量，如饱和度和对比度。在质量度量的指导下，我们从序列中选择好的像素，并将它们组合成最终的结果。

Exposure fusion is similar to other image fusion techniques for depth-of-field extension [19] and photomontage [1]. Burt et al. [4] have proposed the idea of fusing a multi-exposure sequence, but in the context of general image fusion. We introduce a method that can more easily incorporate desired image qualities, in particular those that are relevant for combining different exposures.
曝光融合与其他图像融合技术类似，用于场深扩展[19]和蒙太奇[1]。Burt et al.[4]提出了融合多曝光序列的想法，但是是在一般图像融合的背景下。我们介绍一种方法，该方法可以更容易地结合所需的图像质量，特别是那些与组合不同曝光相关的图像质量。

Exposure fusion has several advantages. First of all, the acquisition pipeline is simplified, no in-between HDR image needs to be computed. Since our technique is not physically-based, we do not need to worry about calibration of the camera response curve, and keeping track of each photograph’s exposure time. We can even add a flash image to the sequence to enrich the result with additional detail. Our approach merely relies on simple quality measures, like saturation and contrast, which prove to be very effective. Also, results can be computed at near-interactive rates, as our technique mostly relies a pyramidal image decomposition. On the downside, we cannot extend the dynamic range of the original pictures, but instead we directly produce a well-exposed image for display purposes.
曝光融合有几个优点。首先，简化了采集管道，不需要计算中间的HDR图像。由于我们的技术不是基于物理的，所以我们不需要担心相机响应曲线的校准，以及跟踪每张照片的曝光时间。我们甚至可以在序列中添加一个flash图像，以增加额外的细节来丰富结果。我们的方法仅仅依赖于简单的质量度量，如饱和度和对比度，这被证明是非常有效的。此外，由于我们的技术主要依赖于金字塔形图像分解，因此结果可以以近乎交互的速度计算。不利的一面是，我们不能扩展原始图片的动态范围，而是直接为显示目的生成一个曝光良好的图像。

图1 曝光融合演示。多曝光序列直接组装成高质量的图像，无需先转换为HDR。不需要考虑相机特定的知识，比如响应曲线。
整个处理时间只有3.3秒(100万像素)。图片由Jacques Joffre提供。

2.Related Work

High dynamic range (HDR) imaging assembles a high dynamic range image from a set of low dynamic range images that were acquired with a normal camera [5, 17]. The camera-specific response curve should be recovered in order to linearize the intensities. This calibration step can be computed from the input sequence and their exposure settings.

高动态范围(High dynamic range, HDR)成像是从一组用普通相机拍摄的低动态范围图像中合成的高动态范围图像[5,17]。为了使强度线性化，需要恢复相机特有的响应曲线。这个校准步骤可以从输入序列及其曝光设置计算出来。

Most display devices have a limited dynamic range and cannot directly display HDR images. To this end, tone mapping compresses the dynamic range to fit the dynamic range of the display device [26]. Many different tone mapping operators have been suggested with different advantages and disadvantages. Global operators apply a spatially uniform remapping of intensity to compress the dynamic range [7, 14, 24]. Their main advantage is speed, but sometimes fail to reproduce a pleasing image. Local tone mapping operators apply a spatially varying remapping [6, 8, 10, 15, 25, 29], i.e., the mapping changes for different regions in the image. This often yields more pleasing images, even though the result may look unnatural sometimes. The operators employ very different techniques to compress the dynamic range: from bilateral filtering [8], which decomposes the image into edge-aware low and high frequency components, to compression in the gradient domain [10]. The following two local operators are related to our method. Reinhard et al. [25] compute a multi-scale measure that is related to contrast and rescales the HDR pixel values accordingly. This is in a way similar to our measures. However, our measures are solely defined per pixel. The method by Li et al. [15] uses a pyramidal image decomposition, and attenuate the coefficients at each level to compress the dynamic range. Our method is also pyramid-based, but it works on the coefficients of the different exposures instead of those of an in-between HDR image. Other tone mappers try to mimic the human visual system, e.g., to simulate temporal adaptation [20]. Instead, we aim at creating pleasing images and try to reproduce as much detail and color as possible.
大多数显示设备动态范围有限，无法直接显示HDR图像。为此，色调映射压缩动态范围以适应显示设备的动态范围[26]。人们提出了许多不同的色调映射算子，它们各有优缺点。全局算子采用空间均匀的强度映射来压缩动态范围[7,14,24]。它们的主要优点是速度快，但有时不能再现令人愉快的图像。局部色调映射算子应用空间变化的映射[6,8,10,15,25,29]，映射随着图像中不同区域的变化而变化。这通常会产生更令人愉悦的图像，尽管有时结果看起来可能不自然。操作人员使用了非常不同的技术来压缩动态范围:从双边滤波[8](将图像分解为边缘感知的低频和高频分量)到梯度域[10]的压缩。下面两个局部操作符与我们的方法有关。Reinhard等人计算了一个与对比度相关的多尺度测度，并相应地调整HDR像素值。这在某种程度上与我们的测量方法相似。然而，我们的度量是按像素单独定义的。Li等人提出的[15]方法采用金字塔形图像分解，对每一层的系数进行衰减，压缩动态范围。我们的方法也是基于金字塔的，但它适用于不同曝光的系数而不是中间HDR图像的系数。其他色调映射试图模仿人类视觉系统，例如，模拟时间适应[20]。相反，我们的目标是创造令人愉悦的图像，并试图再现尽可能多的细节和颜色。

图2 曝光融合是由每个输入图像的权值映射来指导的。高权重意味着一个像素应该出现在最终图像中。这些权重反映了所需的图像质量，如高对比度和饱和度。图片由Jacques Joffre提供。

Image fusion techniques have been used for many years. For example, for depth-of-field enhancement [19, 13], multimodal imaging [4], and video enhancement [23]. We will use image fusion for creating a high quality image from bracketed exposures. In the early 90’s, Burt et al. [4] have already proposed to use image fusion in this context. However, our method is more flexible by incorporating adjustable image measures, such as contrast and saturation. Goshtasby [11] also proposed a method to blend multiple exposures, but it cannot deal well with object boundaries. A more thorough discussion of these techniques is presented in Sec. 3.3.

图像融合技术已经应用多年。例如，对于场深增强[19,13]，多模态成像[4]，视频增强[23]。我们将使用图像融合从包围曝光中创建高质量图像。在90年代初，Burt等人[4]已经提出在这种情况下使用图像融合。然而，通过结合可调图像测量，例如对比度和饱和度，我们的方法更灵活。Goshtasby[11]还提出了一种混合多重曝光的方法，但不能很好地处理对象边界。第3.3节对这些技术进行了更深入的讨论。

Grundland et al. [12] cross-dissolve between two images using a pyramid decomposition [3]. We use a similar blending strategy, but employ different quality measures.

Grundland等[12]使用金字塔分解在两个图像之间交叉溶解[3]。我们使用类似的混合策略，但采用不同的质量措施。

We demonstrate that our technique can be used as a simple way to fuse flash/no-flash images. Previous techniques for this are much more elaborate [9, 2] and are specifically designed for this case, whereas our method is flexible enough to handle that case as well.

我们演示了我们的技术可以作为一种简单的方法来融合flash/no-flash图像。以前的技术要复杂得多[9,2]且专门针对这种情况设计的，而我们的方法也足够灵活，可以处理这种情况。

3.Exposure Fusion

Exposure fusion computes the desired image by keeping only the “best” parts in the multi-exposure image sequence. This process is guided by a set of quality measures, which we consolidate into a scalar-valued weight map (see Fig. 2). It is useful to think of the input sequence as a stack of images. The final image is then obtained by collapsing the stack using weighted blending.

曝光融合通过保持多曝光图像序列中最好的部分来计算所需的图像。这个过程是由一组质量度量来指导的，我们将这些度量合并成一个标量值权重图(见图2)。将输入序列视为一堆图像是有用的。然后通过使用加权混合折叠堆叠来获得最终图像。

As with previous HDR acquisition approaches [17, 5], we assume that the images are perfectly aligned, possibly using a registration algorithm [30].

与以前的HDR获取方法一样[17,5]，我们假设图像是完全对齐的，可能使用了配准算法[30]。

3.1.Quality Measures

Many images in the stack contain flat, colorless regions due to under- and overexposure. Such regions should receive less weight, while interesting areas containing bright colors and details should be preserved. We will use the following measures to achieve this:

由于曝光不足和过度，堆栈中的许多图像都包含扁平、无色区域。这些区域应该承受较少的重量，而包含明亮颜色和细节的有趣区域应该被保留。为此，我们将采取以下措施：

.Contrast: we apply a Laplacian filter to the grayscale version of each image, and take the absolute value of the filter response [16]. This yields a simple indicator C for contrast. It tends to assign a high weight to important elements such as edges and texture. A similar measure was used for multi-focus fusion for extended depth-of-field [19].

.Contrast：我们将拉普拉斯滤波器应用于每个图像的灰度版本，并取滤波器响应的绝对值[16]。这就产生了一个用于对比的简单指示符C。它倾向于给边缘和纹理等重要元素赋予较高的权重。对于扩展的场深[19]，多焦点融合也采用了类似的方法。

.Saturation: As a photograph undergoes a longer exposure, the resulting colors become desaturated and eventually clipped. Saturated colors are desirable and make the image look vivid. We include a saturation measure S, which is computed as the standard deviation within the R, G and B channel, at each pixel.

.Saturation：随着照片曝光时间的延长，最终得到的颜色会变得不饱和，并最终被剪掉。饱和的颜色是理想的，使图像看起来生动。我们包括饱和度测量S，它被计算为每个像素在R、G和B通道内的标准偏差。

.Well-exposedness: Looking at just the raw intensities within a channel, reveals how well a pixel is exposed. We want to keep intensities that are not near zero (underexposed) or one (overexposed). We weight each intensity i based on how close it is to 0.5 using a Gauss curve: , where σ equals 0.2 in our implementation. To account for multiple color channel, we apply the Gauss curve to each channel separately, and multiply the results, yielding the measure E.

.Well-exposureness：仅仅观察通道内的原始强度，就可以揭示一个像素的曝光情况。我们希望保持强度不接近零(曝光不足)或一(曝光过度)。我们对每个强度i进行加权，加权权重为它与高斯曲线接近程度，高斯曲线为：，其中在我们是实现中为0.2。为了考虑多个颜色通道，我们分别对每个通道应用高斯曲线，并将结果相乘，得到测度E。

For each pixel, we combine the information from the different measures into a scalar weight map using multiplication. We opted for a product over a linear combination, as we want to enforce all qualities defined by the measures at once (i.e. like an “AND” selection, as opposed to an “OR” selection, resp.). Similar to weighted terms of a linear combination, we can control the influence of each measure using a power function:

对于每个像素，我们使用乘法将来自不同度量的信息组合成标量权重映射。我们选择了线性组合的结果，因为我们希望立即强制执行由度量定义的所有质量（即，像“AND”选择，而不是“OR”选择，相反）。与线性组合的加权项类似，我们可以使用幂函数控制每个测度的影响：

with C, S and E, being contrast, saturation, and wellexposedness, resp., and corresponding “weighting” exponents ωC, ωs, and ωE. The subscript ij, k refers to pixel (i, j) in the k-th image. If an exponent ω equals 0, the corresponding measure is not taken into account. The final pixel weight Wij,k will be used to guide the fusion process, discussed in the next section. See Fig. 2 for an example of weight maps.

C、S、E为对比度、饱和度、良好的曝光性，相应的加权指数ωC，ωs，ωE。下标ij，k是指第k幅图像中的像素(i, j)。如果一个指数ω= 0,相应的测量是不考虑。最后的像素权重Wij,k将用于指导融合过程，将在下一节讨论。权重图示例见图2。

3.2 Fusion

We will compute a weighted average along each pixel to fuse the N images, using weights computed from our quality measures. To obtain a consistent result, we normalize the values of the N weight maps such that they sum to one at each pixel (i, j):

我们将计算每个像素的加权平均值来融合N个图像，使用从我们的质量度量计算出的权重。为了得到一致的结果，我们对N个权重映射的值进行标准化，使它们在每个像素(i, j)处的和为1：

The resulting image R can then be obtained by a weighted blending of the input images:

然后对输入图像进行加权混合，得到最终的图像R：

(1)

with the k-th input image in the sequence. Unfortunately, just applying Eq. 1 produces an unsatisfactory result. Wherever weights vary quickly, disturbing seams will appear (Fig. 4b). This happens because the images we are combining, contain different absolute intensities due to their different exposure times. We could avoid sharp weight map transitions by smoothing the weight map with a Gaussian filter, but this results in undesirable halos around edges, and spills information across object boundaries (Fig. 4c). An edge-aware smoothing operation using the cross-bilateral filter seems like a better alternative [22, 9]. However, it is unclear how to define the control image, which would tell us where the smoothing should be stopped. Using the original grayscale image as control image does not work well, as demonstrated in Fig. 4d. Also, it is hard to find good parameters for the cross-bilateral filter (i.e., for controlling the spatial and intensity influence).

用表示序列中的第k个输入图像。不幸的是，仅仅应用Eq. 1并不能得到令人满意的结果。只要权重变化迅速，就会出现令人不安的接缝(图4b)。这是因为我们组合的图像由于曝光时间不同，包含不同的绝对强度。我们可以通过使用高斯滤波器平滑权重图来避免权重图的急剧变化，但这会导致边缘出现不希望出现的晕圈，并在对象边界上溢出信息(图4c)。使用跨双边滤波器的边缘感知平滑操作似乎是更好的选择[22,9]。然而，目前还不清楚如何定义控制图像，这将告诉我们应该在哪里停止平滑。使用原始灰度图像作为控制图像效果不佳，如图4d所示。此外，很难找到交叉双边过滤器好的参数(以控制空间和强度的影响)。

To address the seam problem, we use a technique inspired by Burt and Adelson [3]. Their original technique seamlessly blends two images, guided by an alpha mask, and works at multiple resolutions using a pyramidal image decomposition. First, the input images are decomposed into a Laplacian pyramid, which basically contains band-pass filtered versions at different scales [3]. Blending is then carried out for each level separately. We adapt the technique to our case, where we have N images and N normalized weight maps that act as alpha masks. Let the l-th level in a Laplacian pyramid decomposition of an image A be defined as L{A}l, and G{B}l for a Gaussian pyramid of image B. Then, we blend the coefficients (pixel intensities in the different pyramid levels) in a similar fashion to Eq. 1:

为了解决接缝问题，我们使用了一种受Burt和Adelson[3]启发的技术。他们最初的技术是在alpha掩码的引导下无缝地融合两幅图像，并使用金字塔形图像分解在多个分辨率下工作。首先，将输入图像分解为一个拉普拉斯金字塔，该金字塔基本上包含不同尺度[3]的带通滤波版本。然后对每一层分别进行混合。我们将这种技术应用到我们的例子中，我们有N张图像和N个作为alpha掩码的归一化权重映射。将图像A的拉普拉斯金字塔分解中的第l级定义为，图像B高斯金字塔第l级定义为，然后，我们以类似于式1的方式混合系数(不同金字塔级别的像素强度)：

each level l of the resulting Laplacian pyramid is computed as a weighted average of the original Laplacian decompositions for level l, with the l-th level of Gaussian pyramid of the weight map serving as the weights. Finally, the pyramid L{R}l is collapsed to obtain R. An overview of our technique is given in Figure 3.

得到的拉普拉斯金字塔的每个等级l被计算为等级l的原始拉普拉斯分解的加权平均，其中权重图的高斯金字塔的第l等级用作权重。最后，对金字塔L{R} L进行折叠，得到R。图3给出了我们的技术概述。

图3 我们使用图像的拉普拉斯分解和权重图的高斯金字塔来融合不同曝光的图像，这代表了对比度和饱和度等度量。图片由Jacques Joffre提供。

Multiresolution blending is quite effective at avoiding seams (Fig. 4), because it blends image features instead of intensities. Since the blending equation (1) is computed at each scale separately, sharp transitions in the weight map can only affect sharp transitions appear in the original images (e.g. edges). Conversely, flat regions in the original images will always have negligible coefficient magnitude, and are thus not affected by possibly sharp variations in the weight function, even though the absolute intensities among the inputs could be different there.

多分辨率混合在避免接缝方面非常有效(图4)，因为它混合的是图像特征而不是强度。由于混合方程（1）是分别在每个尺度上计算的，因此权重图中的锐利过渡只会影响原始图像中出现的锐利过渡（例如边缘）。相反，原始图像中的平坦区域将始终具有可忽略的系数幅度，并且因此不受权重函数中可能的急剧变化的影响，即使输入之间的绝对强度可能不同。

For dealing with color images, we have found that carrying out the blending each color channel separately produces good results.

对于彩色图像的处理，我们发现分别进行各个颜色通道的混合会产生很好的效果。

3.3.Discussion

Seamless blending is an often-encountered problem in image processing and graphics. We use a multiresolution technique based on image pyramids [3], but other methods are available as well. In particular, gradient-based blending [21] has shown to be effective, and it has been applied to image fusion as well [1, 23]. Gradient methods convert images to gradient fields first, apply the blending operation, and reconstruct the final image from the resulting gradients. However, reconstruction requires solving a partial differential equation, which can be costly for high resolution images. In addition, gradient-based fusion incurs a scale and shift ambiguity, and is prone to color shifting [23].

无缝混合是图像处理和图形处理中经常遇到的问题。我们使用基于图像金字塔[3]的多分辨率技术，但也有其他方法。特别是基于梯度的混合[21]被证明是有效的，它也被应用于图像融合[1,23]。梯度方法首先将图像转换为梯度场，应用混合操作，并从生成的梯度中重建最终图像。然而，重建需要求解偏微分方程，这对于高分辨率图像来说可能是昂贵的。此外，基于梯度的融合会导致尺度和位移的模糊性，容易发生颜色偏移[23]。

Tone mapping operators may also cause color shifts like oversaturation [15], and possibly reduce contrast (see Fig. 7). Our blending is robust against changes in appearance, as it can be seen as a selection process. Even though we select based on contrast and saturation, we do not directly change pixels to meet these constraints.

色调映射操作符也可能导致颜色移动，如过饱和度[15]，并可能降低对比度(见图7)。我们的混合对于外观的变化是强有力的，因为它可以被视为选择过程。即使我们基于对比度和饱和度进行选择，我们也不会直接更改像素以满足这些约束。

图4 加权融合。输入序列如(a)所示。朴素的逐像素混合(b)由于权重映射的急剧变化而产生明显的接缝。使用高斯核(c)模糊权重图，可以移除接缝，但会在边缘引入晕线。跨双边滤波(d)能够避免一些晕，但不是全部。多分辨率混合(e)创建所需的结果。

Our work bears similarity to early work on image fusion, where the Laplacian (or another) pyramid decomposition is used as well [19, 28, 4]. These methods work directly on the coefficients by retaining only those pyramid coefficient that are most salient. For instance, the coefficients with the largest magnitude are kept [19]:

我们的工作与图像融合的早期工作相似，其中也使用了拉普拉斯（或其他）金字塔分解[19，28，4]。这些方法通过仅保留最显著的金字塔系数直接对系数起作用。例如，保留最大模的系数[19]：

Burt and Kolczynski’s exposure fusion technique [4] is based on the same principle. These approaches compound all details present in the sequence, but they do not necessarily produce an appealing result; see Fig. 5. Instead, we blend the pyramid coefficients based on a scalar weight map, but do not directly process individual coefficients at different levels. Measures like saturation and wellexposedness are hard to evaluate directly from pyramid coefficients. Our technique basically decouples the weighting from the actual pyramid contents, which enables us to more easily define quality measures. In fact, any measure that can be computed per-pixel, or perhaps in a very small neighborhood, is applicable.

Burt和Kolczynski的曝光融合技术[4]也是基于同样的原理。这些方法将序列中出现的所有细节组合在一起，但是它们不一定产生吸引人的结果；参见图5。相反，我们基于标量权重映射混合金字塔系数，但不直接处理不同层次的单个系数。饱和度和曝光度等指标很难直接从金字塔系数中得到评价。我们的技术基本上将权重从实际的金字塔内容中分离出来，这使我们能够更容易地定义质量度量。事实上，任何可以按像素计算的度量，或者在一个非常小的邻域内，都是适用的。

Goshtasby’s technique [11] selects the optimal exposure on a per-block basis, and smoothly blends between blocks. Since blocks may span across different objects, spill information across object boundaries, similar to the artifacts related to Gaussian blurring of the weight map (Fig. 4c).

Goshtasby的技术[11]在每个块的基础上选择最佳曝光，并在块之间平滑地混合。由于块可能跨越不同的对象，所以信息会跨对象边界溢出，这类似于权重图的高斯模糊相关artifacts。

4.Results

All of our examples were constructed from JPG-encoded photographs, with unknown gamma correction and camera response curve. We used equally weighted quality measures (ωC = ωS = ωE = 1) in most examples, except where mentioned otherwise.

我们所有的例子都是由jpg编码的照片构造的，具有未知的伽马校正和相机响应曲线。我们使用同样的加权质量度量(ωC =ωS =ωE = 1)在大多数例子中，提到的除外。

4.1.Quality

Fig. 1 and 2 show a typical bracketed exposure shot: underexposed, normally exposed and overexposed. Every exposure contains relevant information that is not present in the other exposures. Our technique is able to preserve finescale details of the buildings, and the warm appearance of the sky.

图1和图2为典型的包围曝光照片：曝光不足、正常曝光和过度曝光。每一次曝光都包含了其他曝光中不存在的相关信息。我们的技术能够保存建筑的精细细节，以及天空温暖的外观。

图5 与其他基于金字塔的融合技术比较[19,4]。这些方法在输入序列(a)中选择最显著的拉普拉斯金字塔系数，而我们的技术是混合的。结果(b,c)颜色太深，并显示颜色变化。与输入序列(a)相比，我们的技术(e)产生了更可靠的结果。

In Fig. 7 and 9, we compare our result to tone mapping. A rigorous comparison is hard, due to the operators’ implementation-specific differences and parameter settings. We therefore limit ourselves to an informal comparison with a few popular tone mappers. Compared to Durand et al. [8] and Reinhard et al. [25], our method offers better contrast. Li et al.’s approach [15] produces quite similar results to ours in terms of contrast, but it also exhibits slight oversaturation. We had to tweak the saturation parameter in their implementation to correct the colors.

在图7和图9中，我们将结果与色调映射进行比较。由于操作符具体实现的差异和参数设置，很难进行严格的比较。因此，我们仅限于与一些流行的色调映射器进行非正式比较。与Durand et al.[8]和Reinhard et al.[25]相比，我们的方法具有更好的对比度。Li等人的方法[15]在对比度方面的结果与我们的方法非常相似，但也有轻微的过饱和度。我们必须在它们的实现中调整饱和度参数来校正颜色。

表1 我们技术的计算时间。我们计算了1/2、1百万像素和2百万像素图像的结果。N是堆栈中图像的数量。初始化为每个输入图像构建拉普拉斯金字塔。更新步骤计算权重映射、相应的高斯金字塔和混合。对于较小的图像尺寸(半到一百万像素)，用户可以得到交互式的反馈(大约一秒钟)。

The multiresolution blending technique discussed in Sec. 3.2 is not without its problems. In Fig. 6, our result contains a spurious low frequency brightness change, which is not present in the original image set. It is caused by a highly varying change in brightness among the different exposures. Intuitively speaking, this artifact can be considered as a very blurred version of the seam problem, illustrated in Fig. 4b. Constructing a higher Laplacian pyramid partially solves this problem. However, the pyramid height is also limited by the size of the downsampling/upsampling filter [3].

3.2节讨论的多分辨率混合技术并非没有问题。在图6中，我们的结果包含了一个伪低频亮度变化，这在原始图像集中是不存在的。它是由不同曝光之间高度变化的亮度引起的。直观地说，这种伪影可以被认为是接缝问题的非常模糊的版本，如图4b所示。建造一个更高的拉普拉斯金字塔部分地解决了这个问题。然而，金字塔的高度也受到下采样/上采样滤波器[3]大小的限制。

4.2.Performance

Our unoptimized software implementation performs exposure fusion in a matter of seconds; see table 1. After building the Laplacian pyramids, our technique can provide near-interactive feedback (see timings of update step). This enables a user gain more control over the fusion process, as he or she can adjust the weighting of quality measures. Additional controls on the input images, such as linear and non-linear intensity remappings are also possible (like brightness adjustment or gamma curves). This can be used to give certain exposures more influence. Motivated by the work of Strengert et al. [27], we expect that our algorithm could eventually run in real-time on graphics hardware.

我们未优化的软件实现在几秒钟内执行曝光融合；见表1所示。在构建了拉普拉斯金字塔之后，我们的技术可以提供近乎交互式的反馈(参见更新步骤的计时)。这使用户能够更好地控制融合过程，因为他或她可以调整质量度量的权重。对输入图像进行额外的控制，如线性和非线性强度重构(如亮度调整或伽马曲线)也是可能的。这可以用来增加某些曝光的影响。受strongert et al.[27]等人工作的启发，我们期望我们的算法最终能够在图形硬件上实时运行。

4.3.Including Flash-Exposures

A flash exposure can fill in a lot of detail, but tends to produce unappealing images, and it may include spurious highlights and reflections. Recent work on flash photography has introduced techniques for combining flash/no-flash image pairs [9, 22, 2]. Our technique can be used here as well, as our quality measures are also relevant in this case. Fig. 8 shows how our technique has successfully removed a highlight and filled in details, similar to Agrawal et al. [2]. However, it cannot remove flash shadows [9] or unwanted reflections [2].

闪光曝光可以填补很多细节，但往往会产生不吸引人的图像，它可能包括虚假的高光和反射。最近关于flash摄影的研究已经引入了将flash/no-flash图像对组合起来的技术[9,22,2]。我们的技术也可以用在这里，我们的质量测量也与此有关。图8显示了我们的技术如何成功地删除了一个突出部分并填充了细节，类似于Agrawal等人的[2]。但是，它不能删除flash阴影[9]或不需要的反射[2]。

4.4.Comparison of Quality Measures

Fig. 10 shows a comparison of our quality measures. Exposure fusion is performed with either contrast, saturation or well-exposedness. The desk scene in the first row comes out better with saturation turned on. Contrast makes the background a bit dark, and well-exposedness darkens the center of the monitor, making the result look unnatural. For the house scene on the next row, saturation and wellexposedness produce vivid colors, which is less so for contrast. Finally, the last row shows how contrast retains details, which are not present in the saturation image (e.g. in the water, and the buildings’ windows). Well-exposedness yields an interesting image, but it looks less natural than the other two.

图10是我们质量测量的比较。曝光融合是在对比度、饱和度或曝光良好的情况下进行的。第一行的桌面场景在打开饱和度后效果更好。对比度使背景有点暗，而良好的曝光会使显示器中心变暗，使结果看起来不自然。对于下一行的house场景，饱和度和良好曝光会产生生动的色彩，而对于对比度则不那么明显。最后，最后一行显示了对比度是如何保留细节的，这些细节在饱和度图像中是不存在的(例如，在水中和建筑物的窗户中)。良好的曝光会产生一个有趣的图像，但它看起来不像其他两个那么自然。

图6 由于输入图像之间曝光的差异，可能会出现伪低频亮度变化。结果(a)底部显得太亮，与输入图像相比显得不自然。其中一幅输入图像如(b)所示，以供参考。

In general, we found that well-exposedness by itself produces fairly good images. However, in some cases it tends to create an unnatural appearance, because it always favors intensities around 0.5. Saturation and contrast does not have this problem. But then again, the results from those measures are not always as interesting as those produced by well-exposedness.

总的来说，我们发现良好的曝光本身就能产生相当好的图像。然而，在某些情况下，它往往会造成一种不自然的外观，因为它总是倾向于强度在0.5左右。饱和度和对比度没有这个问题。但话又说回来，这些测量的结果并不总是像充分曝光所产生的结果那样有趣。

5.Conclusion

We proposed a technique for fusing a bracketed exposure sequence into a high quality image, without converting to HDR first. Skipping the physically-based HDR assembly step simplifies the acquisition pipeline. It avoids camera response curve calibration, it is computationally efficient, and allows for including flash images in the sequence.

我们提出了一种技术，将包围曝光序列融合成高质量的图像，而无需首先转换为HDR。跳过基于物理的HDR组装步骤可以简化获取管道。它避免了相机响应曲线校准，计算效率高，并允许在序列中包含flash图像。

图7 与几种流行的色调映射技术比较。我们的算法可以产生与其他结果竞争的图像质量。有关更详细的检查，请参见图9。

Our technique blends images in a multi-exposure sequence, guided by simple quality measures like saturation and contrast. This is done in a multiresolution fashion to account for the brightness variation in the sequence. Quality is comparable to existing tone mapping operators. Our approach is controlled by only a few intuitive parameters, which can be updated at near-interactive rates in our unoptimized implementation.

我们的技术以多重曝光顺序混合图像，并通过饱和度和对比度等简单质量指标进行指导。这是以多分辨率方式完成的，以解决序列中的亮度变化。质量可与现有的色调映射操作相媲美。我们的方法仅由几个直观的参数控制，这些参数可以在未优化的实现中以接近交互的速度更新。

We would like to investigate different pyramidal image decompositions, such as wavelets and steerable pyramids. Also, we would like to include more measures, in particular one that would detect camera noise. An optimized GPU implementation would enable the user to interactively control the fusion process, but could also be used to display a multi-exposure video stream [18] in real-time. Finally, we would like to look into the applicability of our technique to other image fusion tasks, such as depth-of-field extension [19] and multimodal imaging [4].

我们想研究不同的金字塔图像分解，如小波和可操纵金字塔。此外，我们还希望包括更多的测量，特别是一个将检测相机噪音。优化后的GPU实现将使用户能够交互式地控制融合过程，但也可以用于实时显示多曝光视频流[18]。最后，我们希望研究我们的技术在其他图像融合任务中的适用性，例如场深扩展[19]和多模态成像[4]。

Acknowledgements

Thanks to Jacques Joffre, Jesse Levinson, Min H. Kim and Agrawal et al. [2] for sharing their photographs. Part of the research at Expertise Centre for Digital Media (EDM) is funded by the European Regional Development Fund.

感谢Jacques Joffre、Jesse Levinson、Min H. Kim和Agrawal等人分享了他们的照片。

数码媒体专业中心(EDM)的部分研究由欧洲区域发展基金资助。