Stanford researchers are using AI to create better VR experiences

Asia's Tech News Daily

Stanford researchers are using AI to create better VR experiences

Virtual and augmented reality headsets are designed to place wearers directly into other environments, worlds and experiences. While the technology is already popular among consumers for its immersive quality, there could be a future where the holographic displays look even more like real life. In their own pursuit of these better displays, the Stanford Computational Imaging Lab has combined their expertise in optics and artificial intelligence. Their most recent advances in this area are detailed in a paper published Nov. 12 in Science Advances and work that will be presented at SIGGRAPH ASIA 2021 in December.

At its core, this research confronts the fact that current augmented and virtual reality displays only show 2D images to each of the viewer’s eyes, instead of 3D – or holographic – images like we see in the real world.

“They are not perceptually realistic,” explained Gordon Wetzstein, associate professor of electrical engineering and leader of the Stanford Computational Imaging Lab. Wetzstein and his colleagues are working to come up with solutions to bridge this gap between simulation and reality while creating displays that are more visually appealing and easier on the eyes.

The research published in Science Advances details a technique for reducing a speckling distortion often seen in regular laser-based holographic displays, while the SIGGRAPH Asia paper proposes a technique to more realistically represent the physics that would apply to the 3D scene if it existed in the real world.

Bridging simulation and reality
In the past decades, image quality for existing holographic displays has been limited. As Wetzstein explains it, researchers have been faced with the challenge of getting a holographic display to look as good as an LCD display.

One problem is that it is difficult to control the shape of light waves at the resolution of a hologram. The other major challenge hindering the creation of high-quality holographic displays is overcoming the gap between what is going on in the simulation versus what the same scene would look like in a real environment.

Previously, scientists have attempted to create algorithms to address both of these problems. Wetzstein and his colleagues also developed algorithms but did so using neural networks, a form of artificial intelligence that attempts to mimic the way the human brain learns information. They call this “neural holography.”

“Artificial intelligence has revolutionized pretty much all aspects of engineering and beyond,” said Wetzstein. “But in this specific area of holographic displays or computer-generated holography, people have only just started to explore AI techniques.”

Yifan Peng, a postdoctoral research fellow in the Stanford Computational Imaging Lab, is using his interdisciplinary background in both optics and computer science to help design the optical engine to go into the holographic displays.

“Only recently, with the emerging machine intelligence innovations, have we had access to the powerful tools and capabilities to make use of the advances in computer technology,” said Peng, who is co-lead author of the Science Advances paper and a co-author of the SIGGRAPH paper.

The neural holographic display that these researchers have created involved training a neural network to mimic the real-world physics of what was happening in the display and achieved real-time images. They then paired this with a “camera-in-the-loop” calibration strategy that provides near-instantaneous feedback to inform adjustments and improvements. By creating an algorithm and calibration technique, which run in real time with the image seen, the researchers were able to create more realistic-looking visuals with better color, contrast and clarity.

The new SIGGRAPH Asia paper highlights the lab’s first application of their neural holography system to 3D scenes. This system produces high-quality, realistic representation of scenes that contain visual depth, even when parts of the scenes are intentionally depicted as far away or out-of-focus.

The Science Advances work uses the same camera-in-the-loop optimization strategy, paired with an artificial intelligence-inspired algorithm, to provide an improved system for holographic displays that use partially coherent light sources – LEDs and SLEDs. These light sources are attractive for their cost, size and energy requirements and they also have the potential to avoid the speckled appearance of images produced by systems that rely on coherent light sources, like lasers.

But the same characteristics that help partially coherent source systems avoid speckling tend to result in blurred images with a lack of contrast. By building an algorithm specific to the physics of partially coherent light sources, the researchers have produced the first high-quality and speckle-free holographic 2D and 3D images using LEDs and SLEDs.

Transformative potential
Wetzstein and Peng believe this coupling of emerging artificial intelligence techniques along with virtual and augmented reality will become increasingly ubiquitous in a number of industries in the coming years.

“I’m a big believer in the future of wearable computing systems and AR and VR in general, I think they’re going to have a transformative impact on people’s lives,” said Wetzstein. It might not be for the next few years, he said, but Wetzstein believes that augmented reality is the “big future.”

Though augmented virtual reality is primarily associated with gaming right now, it and augmented reality have potential use in a variety of fields, including medicine. Medical students can use augmented reality for training as well as for overlaying medical data from CT scans and MRIs directly onto the patients.

“These types of technologies are already in use for thousands of surgeries, per year,” said Wetzstein. “We envision that head-worn displays that are smaller, lighter weight and just more visually comfortable are a big part of the future of surgery planning.”

“It is very exciting to see how the computation can improve the display quality with the same hardware setup,” said Jonghyun Kim, a visiting scholar from Nvidia and co-author of both papers. “Better computation can make a better display, which can be a game changer for the display industry.”

Research Report: “Speckle-free holography with partially coherent light sources and camera-in-the-loop calibration”

Internet Explorer Channel Network
Asia's Tech News Daily
News Related

OTHER NEWS

Reshaping the plastic lifecycle into a circle

In 1950, 2 million metric tonnes of new plastic was produced globally. In 2018, the world produced 360 million metric tonnes of plastics. Because of their low cost, durability and ... Read more »

Dragons-Eye View

It can be hard to appreciate that a human-made, football-pitch-sized spacecraft is orbiting 400 km above our heads, but there it is. The jewel of human cooperation and ingenuity that ... Read more »

430-foot asteroid expected to swipe past Earth on Monday

An asteroid that measures the same size as the height of the Great Pyramid of Giza, is expected to pass by Earth on Monday afternoon, according to NASA. The 430-foot ... Read more »

Strong winds power electric fields in the upper atmosphere

What happens on Earth doesn’t stay on Earth. Using observations from NASA’s ICON mission, scientists presented the first direct measurements of Earth’s long-theorized dynamo on the edge of space: a ... Read more »

Arecibo data still has astronomers in a spin

Data collected by the Arecibo Radio Telescope before it collapsed late last year will help astronomers better understand how our local neighbourhood of galaxies formed. Arecibo was the world’s largest ... Read more »

Researchers team up to get a clearer picture of molten salts

Researchers at the Department of Energy’s Oak Ridge, Brookhaven and Idaho national laboratories and Stony Brook University have developed a novel approach to gain fundamental insights into molten salts, a ... Read more »

Molding, patterning and driving liquids with light

Jiming Bao, professor of electrical and computer engineering at the University of Houston, has developed a new fluid that can be cut open by light and demonstrated macroscopic depression of ... Read more »

Geophysicists detect electron dance deep inside the Earth, with a Twist

Most are aware that electrons are negatively charged particles that surround the nucleus of atoms and whose behaviour governs chemical interactions. However, it is less commonly known that electrons come ... Read more »

Combining two cognitive computing nano-elements into one

Researchers at Tohoku University and the University of Gothenburg have established a new spintronic technology for brain-inspired computing. Their achievement was published in the journal Nature Materials on Nov. 29, ... Read more »

Spire Global completes acquisition of exactEarth Ltd

Spire Global has completed its previously announced acquisition of exactEarth Ltd., a leading provider of global maritime vessel data for ship tracking and maritime situational awareness solutions, by way of ... Read more »

Are Water Plumes Spraying from Europa

In 2005, images of a brilliant watery plume erupting from the surface of Saturn’s moon Enceladus captivated the world. The giant column of vapor, ice particles, and organic molecules spraying ... Read more »

Elon Musk: SpaceX faces possible bankruptcy because of engine woes

SpaceX founder Elon Musk told his employees the space company faces a “genuine risk of bankruptcy” because of its struggles in developing its engine for its Starship flights. In an ... Read more »

Russia's Prichal module docks at ISS

Russia’s new docking module Prichal on Friday successfully docked with the International Space Station (ISS), the Russian space agency Roscosmos said. The new addition to the ISS completed an automated ... Read more »

Rocket Lab Announces Neutron Development Update to be Provided on December 2, 2021

Rocket Lab USA will provide the first major update on the development of the Neutron launch vehicle on December 2, 2021 at 8:00 am EST. The virtual event will be ... Read more »

Thales Alenia Space invests in advanced technology for human space flight

Thales Alenia Space, a joint venture between Thales 67% and Leonardo 33%, has inaugurated a new Friction Stir Welding facility at one of its production sites in Turin, which will ... Read more »

Prototype SETI hardware gets first data from VLA

A system designed to provide data from the National Science Foundation’s Karl G. Jansky Very Large Array (VLA) for analysis in the Search for Extraterrestrial Intelligence (SETI) has successfully acquired ... Read more »

Russia's new docking module arrives at ISS

The five-ton Prichal docking module arrived at the International Space Station at 10:19 a.m. EST, propelled by a modified Russian Progress propulsion compartment. They docked to the Nauka module on ... Read more »

Gaia reveals that most Milky Way companion galaxies are newcomers to our corner of space

Data from ESA’s Gaia mission is re-writing the history of our galaxy, the Milky Way. What had traditionally been thought of as satellite galaxies to the Milky Way are now ... Read more »

Eagle professor's space debris removal device receives patent

An Embry-Riddle Aeronautical University professor and his co-inventors patented a device that provides a workable strategy for combating the problem of debris accumulating in space from defunct satellites. Known as ... Read more »

Pulsar Fusion Demonstrates Green Mach-7 rocket in Switzerland

On Saturday 26 November, In the sleepy mountain town of Gstaad – Switzerland, British company Pulsar Fusion demonstrated its latest green hybrid rocket engine. An impressive visual plume effect of ... Read more »

Apply now to the brand new ESA Junior Professional Programme!

Positions are now open for Junior Professionals! Do you have a strong interest in space? Do you aspire to channel your knowledge, interest and experience into a career in the ... Read more »

Carrier rocket takes off from Sichuan province

China launched a Long March 3B carrier rocket early on Saturday morning to send a communication satellite into space, according to China Aerospace Science and Technology Corp, the country’s leading ... Read more »

NASA and industry embrace laser communications

Our televisions and computer screens display news, movies, and shows in high-definition, allowing viewers a clear and vibrant experience. Fiber optic connections send laser light densely packed with data through ... Read more »

New opportunities to study ions in space

Comets have an environment of plasma which contains a large number of ions with low energies. It is necessary to understand these low-energy ions’ properties in order to understand the ... Read more »

Study suggests Sun is likely an unaccounted source of the Earth's water

Curtin University researchers have helped unravel the enduring mystery of the origins of the Earth’s water, finding the Sun to be a surprising likely source. A University of Glasgow-led international ... Read more »

Astronomers discover ancient brown dwarf with lithium deposits intact

A team of researchers at the Instituto de Astrofisica de Canarias (IAC) and the Instituto Nacional de Astrofisica, Optica y Electronica (INAOE), Mexico, has discovered lithium in the oldest and ... Read more »

In the quantum realm, not even time flows as you might expect

A team of physicists at the Universities of Bristol, Vienna, the Balearic Islands and the Institute for Quantum Optics and Quantum Information (IQOQI-Vienna) has shown how quantum systems can simultaneously ... Read more »

Russia launches new docking module to ISS

A Russian Soyuz rocket carrying the new Prichal docking module for the International Space Station blasted off from the Baikonur cosmodrome in Kazakhstan on Wednesday, the Roscosmos space agency said. ... Read more »

Orbital harmony limits late arrival of water on TRAPPIST-1 planets

Seven Earth-sized planets orbit the star TRAPPIST-1 in near-perfect harmony, and U.S. and European researchers have used that harmony to determine how much physical abuse the planets could have withstood ... Read more »

FCC Validates SES Phase I Accelerated C-band Clearing and Relocation Certification

SES, the leader in global content connectivity solutions, announces the U.S. Federal Communications Commission (FCC) has validated the certification of SES’s Phase I accelerated C-band clearing and relocation activities, a ... Read more »

Astroscale space debris removal leaders announce series F raises $109 million

Astroscale Holdings Inc. (“Astroscale”), the market leader in satellite servicing and long-term orbital sustainability across all orbits, today announced it closed its Series F round with additional funding of U.S. ... Read more »

New possibilities for life at the bottom of Oceanic Worlds

In the strange, dark world of the ocean floor, underwater fissures, called hydrothermal vents, host complex communities of life. These vents belch scorching hot fluids into extremely cold seawater, creating ... Read more »

Doing photon upconversion a solid-Crystals that convert light to more useful wavelengths

Solid-solution organic crystals have been brought into the quest for superior photon upconversion materials, which transform presently wasted long-wavelength light into more useful shorter wavelength light. Scientists from Tokyo Institute ... Read more »

Mom and daughter bound for space after winning Virgin Galactic prize

A health coach from Antigua and Barbuda has won two tickets worth almost $1 million to be among Virgin Galactic’s first space tourists — and plans to take the trip ... Read more »

ISRO rife with speculation about human space mission centre, IN-SPACe shift

With speculations and rumours abounding on shifting of its human space mission centre and the headquarters of private sector space regulator Indian National Space Promotion and Authorisation Centre (IN-SPACe), officials ... Read more »

Rocket industrial park put into operation in Wuhan

The China Aerospace Science and Industry Corporation Limited (CASIC) said that it completed the construction of an intelligent satellite production line and a rocket industrial park in the central Chinese ... Read more »

Tianzhou cargo craft to help advance science

Chinese scientists, researchers and engineers now have the opportunity to use the country’s robotic cargo spacecraft to carry out experiments and tests, according to the China Manned Space Agency. The ... Read more »

First LoRa message bounced off the moon

For the first time ever we bounced a LoRa message off the moon on October 5th 2021, using the Dwingeloo radio telescope. This first was achieved by a team consisting ... Read more »

ESA helps Greece to boost its space investments

Ambitious plans to expand the nascent space industry in Greece – enabling the digital transformation of society while creating jobs and generating prosperity – have received a fillip from ESA. ... Read more »

Testing confirms Webb Telescope on track for targeted Dec 22 lLaunch

Engineering teams have completed additional testing confirming NASA’s James Webb Space Telescope is ready for flight, and launch preparations are resuming toward Webb’s target launch date of Wednesday, Dec. 22, ... Read more »
On free-english-test.com you will find lots of free English exam practice materials to help you improve your English skills: grammar, listening, reading, writing, ielts, toeic