A Bokeh Image Generation Technique using Machine Learning

In photography, bokeh is the aesthetic effect appearing in out-of-focus areas. Bokeh is often used to emphasize a subject or to express a beautiful background. However, such bokeh effects can be expressed with cameras equipped with large-aperture lenses but are not easy to realize with small-aperture lenses. The main contribution of this paper is to propose a technique to generate a bokeh image from a single overall in-focus image using a machine learning approach. We propose two types of U-Net based network models, mainly composed of CNN and Transformer. In addition to using two different types of networks, a depth map obtained from the input image and a circular bokeh image are also provided to the network as auxiliary inputs in order to add the bokeh effect by lens aperture, which is difficult to reproduce by machine learning and to distinguish foreground and background accurately. The experimental results show that the proposed method produces bokeh images very close to those taken with a real camera. Furthermore, the quantitative evaluation results show that the proposed method is almost equal to or better than the state-of-the-art machine learning approaches.