Advances in Digital Music Iconography: Benchmarking the detection of musical instruments in unrestricted, non-photorealistic images from the artistic domain

作者：

Matthia Sabatelli, Montefiore Institute
Nikolay Banar, University of Antwerp
Marie Cocriamont, Royal Museums of Art and History, Brussels
Eva Coudyzer, Royal Institute for Cultural Heritage
Karine Lasaracina, Royal Museums of Fine Arts of Belgium, Brussels
Walter Daelemans, University of Antwerp
Pierre Geurts, University of Liège
Mike Kestemont, University of Antwerp

转载来源：Digital Humanities Quarterly, 2021, Volume 15 Number1，http://www.digitalhumanities.org/dhq/vol/15/1/000517/000517.html

在本文中，我们提出了MINERVA，这是视觉艺术领域中第一个用于非真实感、不受限制的图像集合中乐器检测的基准数据集。这一努力是在音乐图像学的学术背景下进行的，这是一个在音乐学和艺术史交叉的跨学科领域。我们对一些最先进的图像分类和目标检测系统进行了基准测试。我们的结果证明了这项任务的可行性，但也突出了这种艺术材料对计算机视觉构成的重大挑战。我们评估了系统的样本外收集，并提供一个解释讨论的假阳性检测。错误分析对触发检测器的上下文线索产生了许多意想不到的见解。例如，围绕儿童和乐器的图像有一些共同的核心属性，比如肢体语言的亲密感。

作者简介：

Matthia Sabatelli 

Matthia Sabatelli is a Ph.D. candidate in Machine Learning at the Department of Electrical Engineering and Computer Science of the University of Liège where he is supervised by Dr. Pierre Geurts. His main research interests revolve around the transferability and scalability of deep neural networks whose generalization properties are studied under the lens of different machine learning paradigms ranging from computer vision to reinforcement learning.

Nikolay Banar 

Nikolay Banar is a Ph.D candidate at the University of Antwerp (Belgium). His scientific interests lie with the intersection of machine learning and humanities.

Marie Cocriamont 

Marie Cocriamont obtained her Masters degree in Musicology at the University of Ghent in 2016. She specialized in the comparison of didactic methods in Classical Arabic music. In 2019 she started working as a scientific assistant at the Royal Museums of Art and History, where she mainly works as an annotator for the research project INSIGHT (Intelligent Neural Systems as InteGrated Heritage Tools).

Eva Coudyzer 

Eva Coudyzer obtained a Master in Art History and Archaeology in 2004 at the Vrije Universiteit Brussel. She worked in documentation centers and collection management services in several cultural organizations in Belgium. In 2009 she started working as a scientific assistant at the Royal Museums of Art and History, specializing in collection management systems. She was coordinator and partner in several national and international digitization projects with a main focus on linking and publishing collections with the use of standardized controlled vocabularies. She currently works as a scientific assistant at the information center of the Royal Institute for Cultural Heritage where she participates in the development of the collection management system and the valorization of the collection in digitization projects.

Karine Lasaracina 

Karine Lasaracina has master degrees in art history and journalism. She joined the RMFAB in 1999 and is now head of the Digital Museum unit. From the very beginning of her career, she has been interested in the concept of digital management of heritage data. A current focus is the development of digital applications that can support enriched visitor experiences in the museum through the implementation of various innovative technological solutions, for example virtual reality tools, multimedia narratives and virtual exhibitions. Promoter of various ongoing research projects, she also works on Data Interoperability, Open Science, the development of Artificial Intelligence to serve the museums, as well as innovation in the field of images of artworks (reproduction, storage, preservation and sharing).

Walter Daelemans 

Walter Daelemans is professor of Computational Linguistics at the University of Antwerp and research director of the CLiPS (Computational Linguistics, Psycholinguistics and Sociolinguistics) research centre. His expertise is in Natural Language Processing and Machine Learning and applications in automatic text analysis and computational stylometry.

Pierre Geurts 

Pierre Geurts is professor in computer science at the University of Liège. His research interests concern the design, the empirical, and the theoretical analyses of machine learning algorithms, with emphasis on scalability, interpretability, and usability of these algorithms. He develops real-world applications of these algorithms in various domains, including computational and systems biology, computer vision, and digital humanities.

Mike Kestemont 

Mike Kestemont is research professor in Digital Text Analysis at the University of Antwerp (Belgium). His expertise lies in the application of computational methods to the Humanities, in particular premodern literature. With F. Karsdorp and A. Riddell, he has co-authored the monograph Humanities Data Analysis: Case Studies with Python, which will appear with Princeton University Press in early 2021.