New Imaging Technique Helps Observe 3-D Hidden Objects through clouds and fog

Researchers at Stanford University have worked with hardware that is analogous to what allows self-driving cars to “see” the realm around them and subsequently improved their vision system with an extremely efficient algorithm that can rebuild 3-D concealed scenes based on the motion of individual photons or particles of light

California:Tests were performed in which the researchers’ system effectively rebuilt shapes hidden by 1-inch-thick foam. But to the human eye, it was similar to viewing through walls. The tests were described in the Nature Communications journal on September 9, 2020.

Gordon Wetzstein, an assistant professor of electrical engineering at Stanford said, “Lot of imaging techniques make images look a little bit better, a little bit less noisy, but this is something where we make the invisible visible. This is pushing the frontier of what may be possible with any kind of sensing system. It’s like superhuman vision.”

The new method complements other kinds of vision systems that can view through barriers on the microscopic scale for use in medicine because the method is more focused on large-scale circumstances such as navigating autonomous cars in heavy rain or fog, and satellite imaging of the surface of planets, including Earth, through the foggy atmosphere.

To view through surroundings that distribute light in every way, the vision system combines a laser with a highly sensitive photon detector that captures every part of laser light striking it.

When the laser scans an obstruction, such as a foam wall, an intermittent photon will manage to travel through the foam, strike the objects that are concealed behind it, and travel back through the foam wall to reach the detector.

The software, supported by the algorithm, subsequently utilizes those few photons and also data about when and where they strike the detector to rebuild the concealed objects in 3-D.

"We were interested in being able to image through scattering media without these assumptions and to collect all the photons that have been scattered to reconstruct the image," said David Lindell, a graduate student in electrical engineering at Stanford.

In the future, a successor of this vision system might be sent through space to moons as well as other planets to help observe deeper surfaces and layers through icy clouds. The investigators would prefer to work with different scattering surroundings to reproduce other situations where the new technology may prove handy.

"You couldn't see through the foam with your own eyes, and even just looking at the photon measurements from the detector, you don't see anything. But, with just a handful of photons, the reconstruction algorithm can expose these objects - and you can see not only what they look like, but where they are in 3-D space,” said Lindell.

Someday, a descendant of this system could be sent through space to other planets and moons to help see through icy clouds to deeper layers and surfaces. In the nearer term, the researchers would like to experiment with different scattering environments to simulate other circumstances where this technology could be useful.

Both Lindell and Wetzstein are also excited about how this latest study signifies a profoundly interdisciplinary intersection of science and engineering.

"These sensing systems are devices with lasers, detectors, and advanced algorithms, which puts them in an interdisciplinary research area between hardware and physics and applied math. All of those are critical, core fields in this work and that's what's the most exciting for me, “said Wetzstein.