Angelic Movement. Exploring and Understanding Art, Iconography and Composition with Machine Learning

The progress of computer vision and deep learning could not be adapted on artworks directly. Even if art represents reality, the perception is different in comparison to photos. Additionally, particular scenes, such as the Annunciation can be given in various settings, compositions, styles and techniques. Thus for art history and other fields analyzing cultural heritage, specific, applied approaches must be developed. We analyze the similarity of whole images across iconographies and we explore the variety inside an iconography. One key in the manifold of representations is the pose. The pose can help to compare compositions and to analyze the narration. Another approach towards image understanding is detecting the objects which form the scene. Convolutional Neural Networks need big data to be trained. This was one reason to focus on christian art and very prominent iconographies like the Annunciation or the Adoration of the Magi and Shepherds. Only with these canonical, ubiquitous scenes could we reach more than 20 000 pictures per iconography. This number not only enables the use of CNNs, it also allows us to see the diversity of motives through the centuries in distant viewing. I want to show how we explore and analyze this data being inspired by art historians like Michael Baxandall, Max Imdahl and obviously Aby Warburg.

Peter Bell (University of Erlangen-Nürnberg)

Projet MemoRekall