Nao Tokui, an enthusiast from Japan, decided that famous art masterpieces would only win if they are followed by a proper background sounds, so he made his own program that uses neural network technology to analyze the pictures and play sounds that are appropriate. For example, if a sea is depicted, it plays sounds of waves; if there are people on it, indistinct voice recordings, and so on.
having fun with cross-modal audio retrieval with famous paintings :-) pic.twitter.com/bqWhVbyivI
— Nao Tokui (@naotokui_en) 13 февраля 2018 г.
READ MORE: AI Generates New Harry Potter Chapter and Netizens Can't Get Enough of It
It didn't require much effort — he used a neural network Soundnet which, by analyzing thousands of videos, learned to detect certain objects on them, and to correspond the visuals to sounds. He also used a free online picture gallery and sound bank.
