New Scientist, UK : The idea came to Jia-Bin Huang at Virginia Tech University when he visited a zoo and was frustrated at his inability to get a good photo of the animals.
He and his colleagues developed their AI to clean up such images. Its neural network analyses several frames in a video, or in a "motion photo" taken by some smartphones, and identifies objects in the images. It then uses any slight change of angle between frames to calculate the distance to each object.
"When you move the camera, you'll find the motion for each layer will be different because they're at different depths," says Huang.
The algorithm then separates out the objects into different layers and removes the foreground layer, providing an unobstructed view of the objects behind (arxiv.org/abs/2004.01180).
Removing objects from images isn't new, but this method of doing so is. Rather than labeling certain features as things to be removed or not, the neural network automatically discovers the distracting foreground objects in the process of learning.
"What this paper does really well and nicely is to use the same approach to resolve multiple image-enhancement problems, like removing reflections from windows as well as a fence obstructing a view behind," says Dima Damen at the University of Bristol, UK, who calls it a "seminal paper in the field".
Huang hopes to make further improvements to the tool and to run it on smartphones.
Also Read: Create your own moody quarantine music using Lo-Fi Player, created by the Google Magenta team
(c) 2020 New Scientist Ltd.
Distributed by Tribune Content Agency, LLC
