Additionally, with the proliferation of low-cost cameras, i.e., point-and-shoots and ... For instance, other types of do
6
Conclusions and Future Work “I may not have gone where I intended to go, but I think I have ended up where I intended to be.” —Douglas Adams
Photography is integral to many aspects of business and home life. Cameras are increasingly important in areas from automotives, to medicine, security, and entertainment. As a result, photography is being used in new scenarios and by new users whose needs are not best met by current photographic methods. In many cases photographs lack the quality needed for a desired application. Additionally, with the proliferation of low-cost cameras, i.e., point-and-shoots and camera-phones, combined with the significant growth in the number of casual photographers, there is a strong need for simple, automatic, and accurate methods to correct image artifacts. In this dissertation, we have explored the problem of image correction and enhancement by using image models that incorporate prior information. In contrast with previous work that has used generic image priors, we presented methods that use priors and models that are tuned to the content of a specific image. We presented three areas of work. We first discussed “PSF Estimation using Sharp Edge Prediction”, where by making the under-lying assumption that all edges in a sharp image are step-edges, our algorithm predicts the ”sharp” version of a blurry input image and uses the two images together to solve for a PSF. We then discussed “Image Enhancement using Color Statistics”, where we have investigated using local-color statistics of an image to improve the debluring, denoising, up-sampling,
131
132
and demosaicing of images using a single framework. Lastly, we discussed “Image Correction using Identity-Specific Priors”, where we developed methods that use identity-specific example images to provide the guidance needed to perform deblurring, up-sampling, and color and exposure adjustments automatically. While there has been significant advancement in photography in recent years, the majority of photographers continue to follow a traditional process: photos are taken one at a time with a single camera, developed and/or processed using extensive manual methods or relatively rudimentary automatic techniques, and viewed as static images. Research in computer vision and graphics has begun to break this mold, and in this dissertation we have addressed some specific aspects of this endeavor. There are several future directions for related work, and there are three specific high-level directions that seem promising.
6.0.1 Building more “Intelligence” into the Photographic Process Traditionally, photography is a serial process consisting of image acquisition, processing, and display. As photographers become more experienced, they learn and refine their process and the quality of their images improve, but the process itself is un-evolving and memory-less. A promising area for further research is how to create a processing pipeline that learns from the results generated by a particular photographer or from the properties of more general image collections. This dissertation has touched on a several facets of this; however, there are interesting avenues for future work. For instance, other types of domain specific knowledge could be used for image correction, such as priors or models for different types of images, i.e. portraits vs. text vs. landscapes. Alternatively, more extensive machine learning approaches could be used to model a particular photographer’s preferences and habits. It would be interesting to leverage information from programs, such as Adobe Lightroom, that store an image’s entire edit history. One could extract this edit history to create a textual description of the edits. This labeled set of edits and the actual image-space changes could be a rich set of information for a machine learning approach that attempts to model when a particular correction is needed.
133
6.0.2 Video Enhancement using Content Specific Priors The work in this dissertation has focused on still images, yet videos suffer from many similar artifacts. However, while many artifacts are similar, extensions of still image methods to include a temporal component for video are not trivial, as there is a need to maintain frame-toframe consistency to avoid temporal flickering and jumping artifacts. An interesting direction is to adapt and extend the work in this dissertation from still image corrections to video. For example, the two-color model discussed in Chapter 4 could be constructed from both a spatial and temporal neighborhood, which could have advantages for temporal denoising and deblurring. Due to the large amounts of data and time involved in processing video, automatic correction methods have even more potential to improve current the video processing pipelines.
6.0.3 Enhancement using Images and Video Another aspect of the digital camera boom is not only are there more still cameras in peoples’ hands, but there are also many more video cameras, as virtually all point-and-shoots and increasingly more camera-phones have video modes. As it becomes easier to acquire both forms of these media in quick succession or even simultaneously, their are increasing opportunities for combined image and video correction that could be applied to either medium.
