Nine Ways We Use AR and VR on the International Space Station

Nine Ways We Use AR and VR on the International Space Station

Even the most highly trained and experienced person sometimes needs a hand. For astronauts aboard the International Space Station, that helping hand comes from other crew members, experts on the ground, and increasingly, in the form of augmented reality (AR) and virtual reality (VR).

The first use of AR on station, a set of high-tech goggles called Sidekick, provided hands-free assistance to crew members using high-definition holograms that show 3D schematics or diagrams of physical objects as they completed tasks. It included video teleconference capability to provide the crew with direct support from flight control, payload developers, or other experts.

The ways in which crew members use these tools continues to expand in frequency and scope. Here are nine ways we use AR and VR for research aboard the space station:

1. VR control of robots

Pilote, an investigation from ESA (European Space Agency) and France’s National Center for Space Studies (CNES), tests remote operation of robotic arms and space vehicles using VR with interfaces based on haptics, or simulated touch and motion. Results could help optimize the ergonomics of workstations on the space station and future spacecraft for missions to the Moon and Mars.

Pilote compares existing and new technologies, including those recently developed for teleoperation and others used to pilot the Canadarm2 and Soyuz spacecraft. The investigation also compares astronaut performance on the ground and during long-duration space missions. Designs from Earth-based testing use ergonomic principles that do not fit microgravity conditions, which is why testing is performed in space.

2. Pedaling through space

Immersive Exercise tests whether a VR environment for the station’s exercise bicycle, CEVIS, increases motivation to exercise and provides astronauts a better experience for their daily training sessions. If crew members enjoy the VR experience, a headset and associated environments (cycling around a lunar crater, anyone?) could become a permanent fixture for exercise sessions. On Earth, this type of VR could benefit people in confined or isolated places.

3. AR maintenance assists

T2 AR tests using AR to help crew members inspect and maintain the space station’s T2 Treadmill. Astronaut Soichi Noguchi of the Japan Aerospace Exploration Agency (JAXA) kicked off the first of a series of tests in April. On future space missions, crew members need to be ready perform this type of task without assistance from the ground due to significant time delays in communications. AR guidance on complex spacecraft maintenance and repair activities also reduces the time astronauts spend training for and completing such tasks. Acting as a smart assistant, AR applications run on tablets or headsets, interpreting what the camera sees and what a crew member does and suggesting the next step to perform. Crew members can operate the applications by speaking or gesturing as well.

4. A cool upgrade

NASA’s Cold Atom Lab (CAL) is the first quantum science laboratory in Earth orbit, hosting experiments that explore the fundamental behaviors and properties of atoms. About the size of a minifridge, it was designed to enable in-flight hardware upgrades. In July, the Cold Atom Lab team successfully demonstrated using an AR headset to assist astronauts with upgrade activities.

Astronaut Megan McArthur donned a Microsoft HoloLens while she removed a piece of hardware from inside CAL and replaced it with a new one. Through the HoloLens, McArthur could easily see the U.S. Destiny module around her. A small front-facing camera on the headset allowed Cold Atom Lab team members to see what she was seeing, whereas normally they would rely on a camera positioned behind or above the astronaut to provide an often-obscured view of the CAL instrument. The CAL team also could add virtual graphics, such as text or drawings, to McArthur’s field of view. For example, as she looked at a large cable harness, the team could add an arrow in her field of view designating a particular cable to unplug or a zip tie to cut.

5. Time travel

Astronauts need to accurately perceive time and speed of objects in their environment in order to reliably perform tasks. Research shows that our perceptions of time and space overlap, and the speed of the body’s movement may affect time perception. Other factors of spaceflight that can affect time perception include disrupted sleep and circadian rhythms and stress.

Time Perception examines changes in how humans perceive time during and after long-duration exposure to microgravity. Crew members wear a head-mounted VR display, listen to instructions, and use a finger trackball connected to a laptop to respond. They take tests once a month during flight, as well as before launching to space and after returning to Earth, to evaluate adaptive changes.

6. Just like being there

The ISS Experience is an immersive VR series filmed over multiple months to document different crew activities, from science conducted aboard the station to a spacewalk. The series uses special 360 cameras designed to operate in space to transport audiences to low-Earth orbit and make viewers feel like astronauts on a mission. It also gives audiences on Earth a better sense of the challenges of adaptation to life in space, the work and science, and the human interaction between astronauts. It could spark ideas for research or programs to improve conditions for crew members on future missions and inspire future microgravity research that benefits people on Earth.

7. Get a better grip

The way humans grip and manipulate an object evolved in the presence of gravity. Microgravity changes the cues we use to control these activities. An ESA investigation, GRIP, studies how spaceflight affects gripping and manipulation of objects. Results could identify potential challenges astronauts may face when they move between environments with different levels of gravity, such as going from the microgravity of a lengthy voyage in space to the surface of Mars, where gravity is about one-third that on Earth. The study also could contribute to the design and control of touch-based interfaces such as remote control on future space exploration.

8. Controlling movement in microgravity

To control the movement and position of our bodies and evaluate the distance between our bodies and other things, humans combine what we see, feel, and hear with information from the inner ear or vestibular system. VECTION looks at how changes in gravity affect these abilities in astronauts and how to address any issues that result. Data collection at multiple time points during flight on the space station and after return to Earth allows researchers to investigate how astronauts adapt to and recover from these effects. This investigation also could help drivers, pilots, and robotic manipulators control vehicles in low-gravity environments.

9. An astronaut’s reach should not exceed their grasp

When a person reaches out to grab an object, their brain coordinates hand movement with information from senses such as sight and hearing. The ESA’s GRASP investigation seeks to better understand the role that perception of gravity plays in this action by observing astronauts using a VR headset to reach for virtual objects. To live in space, astronauts must adapt physically to microgravity, and their brains must adapt to the absence of traditional up and down sensations. By providing insight into this adaptation, GRASP helps support future long-term space exploration.

Internet Explorer Channel Network
News Related


Trading space: ESA bolsters European business

ESA’s orbiting laboratory, OPS-SAT, has hosted the first-ever stock trade in space on Sept 30. The successful experiment required developers at Europe’s leading online broker flatexDEGIRO to think far outside ... Read more »

Euclid telescope ready for extreme space environment

ESA’s Euclid mission has reached a new milestone in its development with successful testing of the telescope and instruments showing that it can operate and achieve the required performance in ... Read more »

Scientists recreate cosmic reactions to unlock astronomical mysteries

Experiments will give scientists a closer look at how exploding stars create world’s heaviest elements. How do the chemical elements, the building blocks of our universe, get built? This question ... Read more »

'Planet confusion' could slow Earth-like exoplanet exploration

When it comes to directly imaging Earth-like exoplanets orbiting faraway stars, seeing isn’t always believing. A new Cornell study finds that next-generation telescopes used to see exoplanets could confuse Earth-like ... Read more »

NASA selects partners for Geostationary and Extended Observations Sounder Phase A Studies

NASA has selected Ball Aerospace and Technologies Corporation of Boulder, Colorado and L3Harris Technologies Inc. of Fort Wayne, Indiana, for a Geostationary and Extended Observations (GeoXO) Sounder (GXS) Phase A ... Read more »

NASA's Lucy science mission will fly by eight asteroids

NASA plans to launch its Lucy spacecraft from Florida on Oct. 16 to fly by eight asteroids starting in 2025, marking the first time scientists will gain close-up views of ... Read more »

Earth from Space: Mackenzie River, Canada

The Copernicus Sentinel-1 mission takes us over the Mackenzie River, a major river system in the Canadian boreal forest. Its basin is the largest in Canada and is the second ... Read more »

BepiColombo swings by Mercury ahead of 2025 orbital insertion

The joint European-Japanese BepiColombo mission captured this view of Mercury on 1 October 2021 as the spacecraft flew past the planet for a gravity assist manoeuvre. The image was taken ... Read more »

Lucy mission prepares for launch to Trojan asteroids

NASA has tested the functions of Lucy, the agency’s first spacecraft to study Jupiter’s Trojan asteroids, filled it with fuel, and is preparing to pack it into a capsule for ... Read more »

Investigating the potential for life around the galaxy's smallest stars

When the world’s most powerful telescope launches into space this year, scientists will learn whether Earth-sized planets in our ‘solar neighborhood’ have a key prerequisite for life – an atmosphere. ... Read more »

NASA confirms Roman Mission's flight design in milestone review

NASA’s Nancy Grace Roman Space Telescope has successfully passed its critical design review, signaling that all design and developmental engineering work is now complete. “After seeing our extensive hardware testing ... Read more »

Earth is dimming due to climate change

Warming ocean waters have caused a drop in the brightness of the Earth, according to a new study. Researchers used decades of measurements of earthshine – the light reflected from ... Read more »

NASA awards Sun-Sky Scanning Sun Photometers for the AERONET Project

NASA has awarded a contract for Sun-Sky Scanning Sun Photometers to CIMEL Electronique of Paris, France to support the AErosol RObotic NETwork (AERONET). This is a firm-fixed-price, indefinite delivery/indefinite quantity ... Read more »

Space technology rocketing upwards, reports IDTechEx

Space. The final frontier, the first step in a journey just beginning, or a horrifying void matched only by your deep fear of the ocean? Whatever your feelings on space, ... Read more »

Extending LIGO's reach into the cosmos

Since LIGO’s groundbreaking detection, in 2015, of gravitational waves produced by a pair of colliding black holes, the observatory, together with its European partner facility Virgo, has detected dozens of ... Read more »

Optically generated quantum fluids of light reveal exotic matter-wave states in condensed matter physics

Researchers from Skoltech and the University of Southampton, U.K., used all-optical methods to create an artificial lattice whose nodes house polaritons – quasiparticles that are half-light and half-matter excitations in ... Read more »

Blue Origin accused of 'toxic' work culture, compromising safety

A group of current and former Blue Origin employees on Thursday accused Jeff Bezos’ space company of having a “toxic” work culture with rampant sexual harassment and a pattern of ... Read more »

China opens Shenzhou-12 return capsule at ceremony

China held a ceremony Monday to open the capsule of the Shenzhou-12 spacecraft which carried three astronauts back to Earth on Sept. 17. Space officials, including Hao Chun, director of ... Read more »

Soyuz docks to new Nauka module port at ISS

The Soyuz MS-18 spacecraft that first launched and arrived to the International Space Station April 9 has now successfully relocated with its crew aboard from the station’s Earth-facing Rassvet module ... Read more »

Quantum photonics based on metasurfaces

In a new publication from Opto-Electronic Advances researchers led by Professor Shuming Wang from Nanjing University, Nanjing, China discuss quantum photonics based on metasurfaces. This paper summarizes recent works on ... Read more »

Planets gone rogue could sustain life

A rogue planet is an interstellar object of planetary mass without a host planetary system. As they freely roam around space, could they be fertile nurseries for life? A Florida ... Read more »

Bare Super-Earths offer clues to evolution of hot atmospheres

A group of astronomers from the Astrobiology Center, the National Astronomical Observatory of Japan, the University of Tokyo, and other institutes, discovered two rocky super-Earth exoplanets lacking thick primordial atmospheres ... Read more »

Beam diagnostics for future laser wakefield accelerators

The principle of laser wakefield accelerators: A high-power laser excites a charge wave in a plasma, which propagates at the speed of the laser pulse and pulls electrons behind it ... Read more »

NASA readies for future Artemis Moon Missions with rocket engine test series

NASA marked a significant milestone Sept. 30 in its plans for future missions to the Moon and, eventually, Mars with completion of an RS-25 single-engine Retrofit-2 test series at Stennis ... Read more »

Ariane 6 launch complex inaugurated at Europe's Spaceport

The new launch complex built for Europe’s upcoming Ariane 6 rocket is inaugurated at Europe’s Spaceport in French Guiana. With this, ESA celebrates another important milestone in the Ariane 6 ... Read more »

China launches high-resolution Earth-observation satellite

China launched a Kuaizhou 1A carrier rocket to place a high-resolution Earth-observation satellite in space on Monday afternoon, according to China Aerospace Science and Industry Corporation. The State-owned space contractor ... Read more »

Hyperbolic Mirrors for Earth Observation Satellites

Optical Surfaces Ltd is a leading manufacturer of high precision hyperbolic mirrors that are used in satellite-based telescopes to collect and focus light enabling high-resolution remote observation in applications including ... Read more »

Asteroid sample brought back to Earth gets a close-up look at Brown

In December 2020, Japan’s Hayabusa2 spacecraft swung by Earth to drop off a cache of rock samples taken from a near-Earth asteroid called Ryugu. Asteroids like Ryugu are thought to ... Read more »

Space for our Planet: Space Solutions for a Sustainable World

At ESA, we believe that we have a responsibility to use our space technologies, applications and services to benefit planet Earth and humankind. Some examples of how we do this ... Read more »

FAA clears Virgin Galactic to resume flights after investigation

The Federal Aviation Administration said Wednesday that Richard Branson’s space tourism company can return to spaceflight after conducting an investigation into issues with the firm’s July 11 launch. A probe ... Read more »

National Geospatial-Intelligence Agency funds Phase 4a of MagQuest Challenge

The National Geospatial-Intelligence Agency (NGA) just launched the Demonstration Phase (Phase 4a) of its MagQuest Challenge to develop novel data collection approaches for the World Magnetic Model. The WMM ultimately ... Read more »

Did a cosmic impact destroy an ancient city in the Jordan Valley

In the Middle Bronze Age (about 3600 years ago or roughly 1650 BCE), the city of Tall el-Hammam was ascendant. Located on high ground in the southern Jordan Valley, northeast ... Read more »

Starfish Space raises $7M to develop Space Tug

Starfish Space, a satellite servicing company founded by former Blue Origin engineers, has raised a $7M funding round co-led by NFX and MaC Venture Capital, with participation from PSL Ventures, ... Read more »

Microscopic metavehicles powered by nothing but light

Researchers from Chalmers University of Technology, Sweden, have succeeded in creating tiny vehicles powered by nothing but light. By layering an optical metasurface onto a microscopic particle, and then using ... Read more »

NASA to launch climate change-tracking Landsat 9 satellite

NASA plans to launch one of its most high-tech Earth observation satellites to date Monday from California to help track climate events that range from California wildfires to deforestation of ... Read more »

Gigantic cavity in space sheds new light on how stars form

Astronomers analyzing 3D maps of the shapes and sizes of nearby molecular clouds have discovered a gigantic cavity in space. The sphere-shaped void, described in The Astrophysical Journal Letters, spans ... Read more »

How to weigh a quasar

Astronomers of the Max Planck Institute for Astronomy have, for the first time, successfully tested a new method for determining the masses of extreme black holes in quasars. This method ... Read more »

Satellite maker Terran Orbital plans major plant in Florida

Small-satellite maker Terran Orbital plans to build a large manufacturing plant with more than 2,000 employees near Kennedy Space Center’s former space shuttle landing strip, the company and Florida officials ... Read more »

NASA adviser blasts lack of congressional action on space traffic dangers

The chair of NASA’s independent safety panel blasted Congress on Thursday for not designating a federal agency to spearhead space traffic management. Chairwoman Patricia Sanders, a former Department of Defense ... Read more »

Cloud-spotting on a distant exoplanet

An international team of astronomers has not only detected clouds on the distant exoplanet WASP-127b, but also measured their altitude with unprecedented precision. A presentation by Dr Romain Allart at ... Read more »