OSCAR: On-Site Composition and Aesthetics ... - Stanford InfoLab

We multiply the matching function by 2 to encourage matching of diagonal categories in practice. Note that the value of si is between 0 and 6, because one image can at most be classified ...... 3 ,f2 was constructed, each polynomial represents the features. Linear regression was preferred to Support Vector Regression in ...
4MB Sizes 0 Downloads 74 Views
Noname manuscript No. (will be inserted by the editor)

OSCAR: On-Site Composition and Aesthetics Feedback through Exemplars for Photographers Lei Yao · Poonam Suryanarayan · Mu Qiao · James Z. Wang · Jia Li

Received: date / Accepted: date

Abstract In this paper we describe a comprehensive system to enhance the aesthetic quality of the photographs captured by the mobile consumers. The system, named OSCAR, has been designed to provide onsite composition and aesthetics feedback through retrieved examples. We introduce three novel interactive feedback components. The first is the composition feedback which is qualitative in nature and responds by retrieving highly aesthetic exemplar images from the corpus which are similar in content and composition to the snapshot. The second is the color combination feedback which provides confidence on the snapshot to contain good color combinations. The third component is the overall aesthetics feedback which predicts the aesthetic ratings for both color and monochromatic images. An existing algorithm is used to provide ratings for color images, while new features and a new model are developed to treat monochromatic images. This system was designed keeping the next generation photography needs in mind and is the first of its kind. The feedback rendered is guiding and intuitive in nature. It is comLei Yao, Poonam Suryanarayan College of Information Sciences and Technology, The Pennsylvania State University, University Park, PA E-mail: [email protected] Mu Qiao Department of Computer Science and Engineering, The Pennsylvania State University, University Park, PA James Z. Wang College of Information Sciences and Technology, The Pennsylvania State University, University Park, PA Office of International Science and Engineering, National Science Foundation, Arlington, VA Jia Li Department of Statistics, Department of Computer Science and Engineering, The Pennsylvania State University, University Park, PA

puted in situ while requiring minimal input from the user. Keywords Mobile · Photo Composition · Aesthetics Rating · Color Analysis · Digital Photography

1 Introduction Interest in the research community on the plausibility of predicting the aesthetic quality of images has increased dramatically over the past few years. It was established in Datta et al. (2006) that photo aesthetics though being subjective can be estimated using a set of images with a general consensus on their aesthetic quality. Mathematical models could be learnt which can predict the aesthetics of any image. Understanding aesthetics can aid many of the applications like summarization of photo collections (Obrador et al. 2010), selection of high quality images for display (Fogarty et al. 2001) and extraction of aesthetically pleasing images for image retrieval (Obrador et al. 2009). It can also be used to render feedback to the photographer on the aesthetics of his/her photographs. Many other applications have been built around suggesting improvisations to the image composition (Bhattacharya et al. 2010; Liu et al. 2001) through image retargeting, and color harmony (Cohen-Or et al. 2006) to enhance image aesthetics. These applications are more off-line in nature. Although they are able to provide useful feedback, it is not on the spot, and requires considerable input from the user. There is no scope for any improvement on the images captured once the user moves away from the location which a professional feedback on-site can accomplish. In order to make image aesthetic quality assessment more dynamic and to reach out for the general pub-


lic with a practical perspective, we developed a system which can provide on-site feedback to the user. Aesthetics of an image is the result of a complex interplay of many factors like the lighting, the subject form, composition, color harmony, etc. We realized the importance of providing feedback on each of the aesthetic primitives separately by which the user infers what aspect of the photograph n