Enthusiast Creates Neural Network-Based Program that Picks Sound For Art Pieces

An author has used a neural network to detect objects depicted in pictures and select sounds from a free database to accompany the art.
Sputnik

Nao Tokui, an enthusiast from Japan, decided that famous art masterpieces would only win if they are followed by a proper background sounds, so he made his own program that uses neural network technology to analyze the pictures and play sounds that are appropriate. For example, if a sea is depicted, it plays sounds of waves; if there are people on it, indistinct voice recordings, and so on.

READ MORE: AI Generates New Harry Potter Chapter and Netizens Can't Get Enough of It

It didn't require much effort — he used a neural network Soundnet which, by analyzing thousands of videos, learned to detect certain objects on them, and to correspond the visuals to sounds. He also used a free online picture gallery and sound bank.

Discuss