Reconstructing the correspondence between early modern facades and their self-similarity by means of weakly supervised learning.
The interdisciplinary Frontier project between computer vision and art history analyses early modern architecture with the help of machine learning and image processing.
We initially base our approach on a small database of four hundred early modern facades. This database contains reproductions of drawings as well as photographs.
In the first step we detect important parts of the architecture like the capitals of the columns and arcs – parts which have an aesthetic and semantic significance. In the progressing second step we try to analyze the self-similarity of the building structure. Therefore we divide binary edge images of the reproductions into patches of different scales. Since structures like windows, pillars and other architectural elements re-occur several times we are looking for related image regions that can be found frequently in a given image. Fast Directional Chamfer Matching is used to measure similarity. We then define a filter based on the activation pattern of all parts, to analyze the occurring structural atoms and their size within the given building. In the third step the performance of the learned parts is then evaluated by searching the informative structures in new images with the same and similar elements.
Therefore we can finally interpret the structure of a single architecture and compare the designs of different buildings and styles.
Funded by the Innovation Fund FRONTIER, Heidelberg University funded by German Research.
Link to project website.
Name and contact of project responsible(s):
Prof. B. Ommer (Interdisciplinary Center for Scientific Computing, Heidelberg University)
Additionally involved scientists and partners
Prof. M. Hesse (Institute for European Art History, Heidelberg University)
Masato Takami, Peter Bell, and Björn Ommer, An Approach to Large Scale Interactive Retrieval of Cultural Heritage, in: Proceedings of the EUROGRAPHICS Workshops on Graphics and Cultural Heritage, EUROGRAPHICS Association, 2014 (accepted, in press).