Computer vision in the study of art:
New rigorous approaches to the analysis of paintings and drawings

David Stork
Ricoh Innovations and Stanford University

ABSTRACT

Rigorous computer vision algorithms have been used to shed light on a number of recent controversies in the study of art. For example, illumination estimation and shape-from-shading methods developed for robot vision and digital photograph forensics can reveal the accuracy and the working methods of masters such as Jan van Eyck and Caravaggio and the question of whether they secretly traced optically projected images. Sophisticated analysis of the lighting over different figures in a realist painting has shown that the figures were painted under different lighting conditions and hence "compositted" into the image. Computer wavelet analysis has been used for attribution of the contributors in Perugino's Holy Family and works of Vincent van Gogh. Computer methods can dewarp the images depicted in convex mirrors depicted in famous paintings such as Jan van Eyck's Arnolfini portrait to reveal new views into artists' studios and shed light on their working methods.

New principled, rigorous methods for estimating perspective transformations outperform traditional and ad hoc methods and yield new insights into the working methods of Renaissance masters. Statistical analysis of the color and shape of brush strokes allow us to digitally "peel away" layers of brush strokes to reveal intermediate stages in the development of paintings, such as van Gogh's self portraits. Sophisticated computer graphics recreations of tableaus allow us to explore "what if" scenarios, and reveal the lighting and working methods of masters such as Caravaggio and Velązquez.

How do these computer methods work? What can computers reveal about images that even the best-trained connoisseurs, art historians and artist cannot? How much more powerful and revealing will these methods become? In short, how is the "hard humanities" field of computer image analysis of art changing our understanding of paintings and drawings?

This profusely illustrated lecture for scholars interested in computer vision, pattern recognition and image analysis will include works by Jackson Pollock, Vincent van Gogh, Jan van Eyck, Hans Memling, Lorenzo Lotto, and several others. You may never see paintings the same way again.

Joint work with Antonio Criminisi, Andrey DelPozo, David Donoho, Marco Duarte, Micah Kimo Johnson, Dave Kale, Ashutosh Kulkarni, M. Dirk Robinson, Silvio Savarese, Morteza Shahram, Ron Spronk, Christopher W. Tyler, Yasuo Furuichi and Lisa Wong.

Bio

Dr. David G. Stork is Chief Scientist of Ricoh Innovations and Consulting Professor of Statistics at Stanford University, where he has held appointments, taught, and sat on dissertation committees frequently over the last 20 years in the departments of Computer Science, Electrical Engineering, Statistics, Psychology and Art and Art History. He is a Fellow of the International Association for Pattern Recognition and founding general chairman of the Optical Society of America's Digital image processing and analysis conference (DIPA). He has published six books/proceedings volumes, including Seeing the Light: Optics in nature, photography, color, vision and holography (Wiley), the leading textbook on optics in the arts, Computer image analysis in the study of art (SPIE), the first volume in this discipline, Pattern Classification (2nd ed.), the world's all-time best-selling textbook in the field, translated into three languages and used in courses in over 250 universities worldwide, and HAL's Legacy: 2001's computer as dream and reality (MIT), the source of his PBS television documentary, 2001: HAL's Legacy.

Dr. Stork is a graduate in physics of the Massachusetts Institute of Technology and the University of Maryland at College Park. He also studied art history at Wellesley College and was Artist-in-Residence through the New York State Council of the Arts. He holds 38 US patents and has published numerous technical papers, including 36 technical papers on computer image analysis of art. Currently, his central research interests lie in the theoretical foundations of joint design of optics and image processing for imaging systems.

Colloquia Series page.