The first metaverse experiments? Look to what's already happening in medicine

Surgeon Shafi Ahmed poses for a photograph wearing a Microsoft HoloLens headset inside his operating theater at the Royal London Hospital on Thursday, Jan. 11, 2018.Bloomberg | Bloomberg | Getty Images

The metaverse, the digital world's Next Big Thing, is touted as the internet domain where animated avatars of our physical selves will be able to virtually do all sorts of interactivities, from shopping to gaming to traveling — someday. Wonks say it could be a decade or longer before the necessary technologies catch up with the hype.

Right now, though, the health-care industry is utilizing some of the essential components that will ultimately comprise the metaverse — virtual reality (VR), augmented reality (AR), mixed reality (MR), and artificial intelligence (AI) — as well as the software and hardware to power their applications. For example, medical device companies are using MR to assemble surgical tools and design operating rooms, the World Health Organization (WHO) is using AR and smartphones to train Covid-19 responders, psychiatrists are using VR to treat post-traumatic stress (PTS) among combat soldiers, and medical schools are using VR for surgical training.

Facebook, Oculus and Covid

Since Facebook — now Meta Platforms — acquired Oculus and its VR headset technology in 2014 for $2 billion, numerous health-care applications have been developed. One of the latest was a collaboration with Facebook Reality Labs and Nexus Studios and the WHO Academy. The organization's R&D incubator designed a mobile learning app for health workers battling Covid-19 worldwide. One of the training courses involves AR to simulate on a smartphone the proper techniques and sequence to put on and remove person protective equipment. With content available in seven languages, the app is built around the needs expressed by 22,000 global health workers surveyed by the WHO last year.

Oculus technology is used at UConn Health, the University of Connecticut's medical center in Farmington, Connecticut, to train orthopedic surgery residents. Educators have teamed with PrecisionOS, a Canadian medical software company that offers VR training and educational modules in orthopedics. Donning Oculus Quest headsets, the residents can visualize in 3-D performing a range of surgical procedures, such as putting a pin in a broken bone. Because the procedure is performed virtually, the system allows the students to make mistakes and receive feedback from faculty to incorporate on their next try.

Meanwhile, as the metaverse remains under construction, “we see great opportunity to continue the work Meta already does in supporting health efforts,” a Meta spokesperson said. “As Meta's experiences, apps and services evolve, you can expect health strategy to play a role, but it's far too soon to say how that might intersect with third-party technologies and providers.”

When Microsoft introduced its HoloLens AR smart glasses in 2016 for commercial development, early adopters included Stryker, the medical technology company in Kalamazoo, Michigan. In 2017, it began harnessing the AR device to improve processes for designing operating rooms for hospitals and surgery centers. Because ORs are shared by different surgical services — from general surgery to orthopedic, cardiac and others — lighting, equipment and surgical tools vary depending on the procedure.

Recognizing the opportunity the HoloLens 2 provided in evolving OR design from 2D to 3D, Stryker engineers are able to design shared ORs with the use of holograms. The MR experience visualizes all of the people, equipment and setups without requiring physical objects or people to be present.

Zimmer Biomet, a Warsaw, Indiana-based medical device company, recently unveiled its OptiVu Mixed Reality Solutions platform, which employs HoloLens devices and three applications — one using MR in manufacturing surgical tools, another that collects and stores data to track patient progress before and after surgery, and a third that allows clinicians to share a MR experience with patients ahead of a procedure.

“We are currently using the HoloLens in a pilot fashion with remote assist in the U.S., EMEA and Australia,” a Zimmer Biomet spokesperson said. The technology has been used for remote case coverage and training programs, and the company is developing software applications on the HoloLens as part of data solutions focused on pre- and post-procedures, the spokesperson said.

Microsoft's holographic vision of the future

In March, Microsoft showcased Mesh, a MR platform powered by its Azure cloud service, which allows people in different physical locations to join 3-D holographic experiences on various devices, including HoloLens 2, a range of VR headsets, smartphones, tablets and PCs. In a blog post, the company imagined avatars of medical students, learning about human anatomy,  gathered around a holographic model and peeling back muscles to see what's underneath.

Microsoft sees many opportunities for its MR tech, and in March secured a $20 billion contract with the U.S. military for its use with soldiers.

In real-world applications of AR medical technology, Johns Hopkins neurosurgeons performed the institution's first-ever AR surgeries on living patients in June. During the initial procedure, physicians placed six screws in a patient's spine during a spinal fusion. Two days later, a separate team of surgeons removed a cancerous tumor from the spine of a patient. Both teams donned headsets made by Augmedics, an Israeli firm, equipped with a see-through eye display that projects images of a patient's internal anatomy, such as bones and other tissue, based on CT scans. “It's like having a GPS navigator in front of your eyes,” said Timothy Witham, M.D., director of the Johns Hopkins Neurosurgery Spinal Fusion Laboratory.

At the University of Miami's Miller School of Medicine, instructors at the Gordon Center for Simulation and Innovation in Medical Education utilize AR, VR and MR to train emergency first-responders to treat trauma patients, including those who have had a stroke, heart attack or gunshot wound. Students practice life-saving cardiac procedures on Harvey, a life-like mannequin that realistically simulates nearly any cardiac disease. Wearing VR headsets, students can “see” the underlying anatomy which is graphically exposed on Harvey.

“In the digital environment, we're not bound by physical objects,” said Barry Issenberg, MD, Professor of Medicine and director of the Gordon Center. Before developing the virtual technology curriculum, he said, students had to physically be on the scene and train on actual trauma patients. “Now we can guarantee that all learners have the same virtual experience, regardless of their geographic location.”

Since it was founded in 1999, the University of Southern California Institute for Creative Technologies (ICT) has developed VR, AI and other technologies to address a variety of medical and mental health conditions. “When I first got involved, the technology was Stone Age,” said Albert “Skip” Rizzo, a psychologist and director for medical virtual reality at ICT, recalling his tinkering with an Apple IIe and a Game Boy handheld console. Today he uses VR and AR headsets from Oculus, HP and Magic Leap.

Rizzo has helped create a VR exposure therapy, called Bravemind, aimed at providing relief from PTS, particularly among veterans of the wars in Iraq and Afghanistan. During exposure therapy, a patient, guided by a trained therapist, confronts his or her trauma memories through simulations of their experiences. Wearing a headset, the patient can be immersed in several different virtual scenarios, including a Middle-Eastern themed city and desert road environments.

“Patients use a keyboard to simulate people, insurgents, explosions, even smells and vibrations,” Rizzo said. And rather than relying exclusively on imagining a particular scenario, a patient can experience it in a safe, virtual world as an alternative to traditional talk therapy. The evidence-based Bravemind therapy is now available at more than a dozen Veterans Administration hospitals, where it has been shown to produce a meaningful reduction in PTS symptoms. Additional randomized controlled studies are ongoing.

As Big Tech continues to build out the metaverse, alongside software and hardware companies, academia and other R&D partners, the health-care industry remains a real-life proving ground. “While the metaverse is still in its infancy, it holds tremendous potential for the transformation and improvement of health care,” wrote Paulo Pinheiro, head of software at Cambridge, U.K.-based Sagentia Innovation on the advisory firm's website. “It will be fascinating to watch the situation unfold.”

Stay connected with Healthy Returns

For a front row seat at CNBC Events, you can hear directly from the visionary executives, innovators, leaders and influencers taking the stage in “The Keynote Podcast.” Listen now, however you get your podcasts.

For more exclusive insights from our reporters and speakers, sign up for our Healthy Returns newsletter to get the latest delivered straight to your inbox weekly.

News Related


Ministop joins French fry war in Japan with larger Bucket Potato as McDonald's reel from potato shortage

Ministop’s Bucket |Potato Ministop has joined the French fries war in Japan by offering its Bucket Potato, which has three times the amount of their X Fried Potatoes for 641 ... Read more »

German government: natural gas is an interim solution in the energy transition

Photo: ANP The use of natural gas is necessary as an interim solution for the energy transition. The German government wrote this in a letter to the European Commission. Berlin ... Read more »

8 Features of The Best Forex Trading Platforms

Retail trading is a robust industry. Around the world, millions of people buy and sell currencies online, and the term “Forex” is now associated with broad portfolios comprising currencies, stocks, ... Read more »

Shell looks at sustainable projects in France with €4 billion in its pocket

According to Van Beurden, Shell is looking at possible investments in French offshore wind farms, charging stations for electric cars and the production of hydrogen and biofuel for aircraft. France ... Read more »

The 21-year-old College Cuber makes $8,000 mosaics of sports stars using cubes

Dylan Sadiq, known as “The College Cuber” charges $8,000 for his mosaics made of cubes. Pro teams including the NBA's Detroit Pistons have purchased the artwork.Courtesy: Dylan Sadiq | The ... Read more »

In Pics: Reliance Industries' recent acquisitions and investments

Reliance Retail Ventures (RRVL), through its wholly-owned subsidiary 7-India Convenience Retail, entered into a master franchise agreement with 7-Eleven, Inc (SEI) for the launch of 7-Eleven convenience stores in India. ... Read more »

Experts seriously doubt whether patent waivers on Covid-19 vaccines will ever come to be

It's been well over a year since a landmark proposal brought the issue of patent waiver for the mRNA Covid vaccine to the spotlight. But many observers don't see that ... Read more »

Shell looks at billions of investments in France

Photo: ANP Shell is looking at investment opportunities worth 4 billion euros in France. That says CEO Ben van Beurden in an interview with the French newspaper Le Figaro. According ... Read more »

Lufthansa would like 40% stake in Italian ITA

Photo: ANP The German aviation group Lufthansa would like to take a 40 percent stake in the Italian airline ITA Airways, the successor of the bankrupt Alitalia. That reports the ... Read more »

Crypto meltdown erases $1 trillion in value Bitcoin down 50% from November peak as risk-off sentiment batters cryptocurrencies

Representations of the Ripple, Bitcoin, Etherum and Litecoin virtual currencies are seen on motherboard. (Reuters File Photo) Bitcoin has now shed more than 50% from its record high in November ... Read more »

Dolo 650 maker: We did not expect this kind of popularity

Dilip Surana, chairman and managing director at Micro Labs Limited, is—with due respect to Adar Poonawalla, CEO, Serum Insitutute of India, and Bharat Biotech chairman Krishna Ella, the companies behind ... Read more »

Premji Invest backed Fabindia Files Papers For 2022 Listing; Joins Apparel Brand IPO Party

Leading ethnic wear and lifestyle retailer Fabindia which is backed by marquee investors including Azim Premji’s investment arm Premji Invest & Nandan Nilekani, has filed draft papers with Sebi to ... Read more »

S. Korea's Hanmi, Celltrion, and Dongbang to produce Merck's COVID-19 pill

Hanmi will manufacture the finished drug and produce ingredients. South Korea’s Hanmi Pharm Co., Celltrion Inc., and Dongbang FTL were among 27 drugmakers chosen to produce low-cost versions of Merck ... Read more »

Former CBI chief Fairbairn joins race to head Channel 4

Dame Carolyn Fairbairn is among the candidates being considered as the next chairman of the state-owned broadcaster as its potential privatisation looms, Sky News learns. Read more »

‘Takeover hunt for department store giant Kohl's: $9 billion bid on the table'

Insiders report this to Bloomberg news agency. The second-largest department store chain in the United States is under pressure from activist shareholders because it would underperform and the company would ... Read more »

Sources: US talks with Qatar about gas supplies to Europe

Photo: ANP The US government is in talks with Qatar about supplies of liquefied natural gas (LNG) to Europe if Russia were to invade Ukraine, causing European gas shortages. According ... Read more »

Hong Kong's home buyers snap up Henderson Land's The Harmonie flats, defying city's worst one-day explosion of new Covid-19 cases

Hong Kong’s property buyers turned out in droves, defying the city’s worst one-day explosion of new Covid-19 cases to snap up apartments at the first new housing project to be ... Read more »

Bonus rain at top US bankers: boss Morgan Stanley goes home with $35 million

The big boss at Morgan Stanley was paid a base salary of $1.5 million in 2021. On top of that came a cash bonus of $8 million and $5 million ... Read more »

Yes Bank posts 77% rise in December quarter profit on lower provisions, higher recoveries

yes-bank-300_36683164 Yes Bank on Saturday reported a 77% year on year growth in December quarter profit at Rs 266 crore as lower provisions and higher loan recoveries drove earnings at ... Read more »

Bloomberg: multi-billion dollar offer for major department store chain Kohl's

Photo: ANP The major American department store chain Kohl’s has received a takeover offer worth about $ 9 billion, converted almost 8 billion euros, from a group of investors. Insiders ... Read more »

Bitcoin rumbles further down: 25% fall in new year

On crypto platform Coindesk, bitcoin sees another 10% of its value evaporate to a position of $35,580 around 8 a.m. (Dutch time). This means that the digital currency has returned ... Read more »

Current economic landscape may not allow government to go for fiscal consolidation immediately: YES Bank chief economist

Read more »

Instagram's new subscription model: How it will aid Indian creators

While the features are being tested in the US, creators in India bet big on Instagram’s new subscription model. Instagram is testing three new subscription features under which followers can ... Read more »

Impact of oil firms' exit from Myanmar limited 'A lot more pressure needed to defeat this junta for good,' says human rights activist

The TotalEnergies logo is seen on a charging station in La Defense on the outskirts of Paris. The French company announced in Friday that it would pull out of Myanmar. ... Read more »

Payments Council urges rollback of no-fee transaction policy for UPI and Rupay debit cards

The industry estimates a loss of Rs 5,500 crore due to the Zero-MDR norm. (Representative image) The Payments Council of India (PCI), the industry body for digital payments, yesterday said ... Read more »

More trouble for PTC India as independent director quits after three board exits in arm over corporate governance issues

After PTC India Financial Services (PFS) came under the scanner with three of its independent directors resigning citing lapses in compliance and corporate governance, an independent director at the parent ... Read more »

Market falls over 3% but BSE smallcap index hits new high, 33 stocks gain 10-44%

After a positive start for the week on January 17 the Indian benchmark indices remained under selling pressure for the next four session’s in the week ended January 21 in-line ... Read more »

Ombudsman: Return to Australia for a DigiD

In the end, thanks to the help of an official Australian government agency, he manages to have his AOW paid out through the SVB. But he would also like to ... Read more »

Hawkish comments from the Fed may further strengthen the dollar which may not bode well for commodities: Kotak Securities' Ravindra Rao

Commodities witnessed strong gains this week despite the dollar strengthening and equities weakening as rising inflationary pressure drew investors, said Ravindra Rao, vice-president, head of commodity research, Kotak Securities. Inflation ... Read more »

As economies limp out of pandemic, here's how CII wants policymakers to steer post-COVID recovery

Representative image Growth appears to be limping back after COVID battered economies around the world, thanks to policy support from governments as well as vaccination drives. However, to ensure that ... Read more »

Transforming a ghost village with hill-centric livelihood and tourism opportunities

Major Gorki Chandola is reviving traditional practices in agriculture, construction and tourism to sustainably develop his village. Photo by Archana Singh. India’s northern state of Uttarakhand, known as the Land ... Read more »

Mining on the shifty sands of Goan politics

Mining machinery at a Goa mine brought to a halt by Supreme Court of India’s February 2018 judgment. Photo by Meera Mohanty/Mongabay. In the run-up to the 2022 state elections, ... Read more »

Clean energy startups innovate on products to aid farmers, rural areas

A three horse power solar panel installed in a village in Rajasthan by Agrivijay for irrigation works. Photo courtesy AgriVijay. Yash Tarwadi, 25, is a chemical engineer based in Surat ... Read more »

Energy major Shell sheds 'Royal Dutch' from name

Royal Dutch Shell | Headquarters: The Hague, Netherlands. Incorporated in the United Kingdom. (Photo: Reuters) Energy giant Shell has dropped from its name the ‘Royal Dutch’ part, which it had ... Read more »

Moneycontrol PRO & Espresso present Nifty Banker 2.0 on Saturday, 22nd January, at 8 pm, with Rohit Katwal on “Momentum Short - Index Option Selling using trend following on Nifty and Bank Nifty”

Nifty Banker 2.0 is India’s First Retail Index Traders Online Conference to be organised in the country. Some of the best index traders in India will share their deep trading ... Read more »

Why Always Hire A Professional and Experienced Tax consultant in vienna (Steuerberater Wien)

Accounting is, above all, strategic management by taking responsibility for drawing up the accounts. It is important to have this vision. Each year, accounting produces mandatory summary documents at the ... Read more »

NetEase apologises for ‘bad experiences' after fans of Harry Potter game revolt over Lunar New Year campaign

Chinese players of a video game based on the Harry Potter series of books by J.K. Rowling have staged an online revolt against NetEase, with their complaints about unfair game ... Read more »

Cryptocurrency crash wipes out more than US$1 trillion in market value

For bitcoin, there’s only been one constant recently: decline after decline after decline. And the superlatives have piled up really quickly. With the Federal Reserve intending to withdraw stimulus from ... Read more »

Moneycontrol Pro Weekender | Bubble Trouble 

Representative image Dear Reader, Jeremy Grantham, GMO’s chief investment strategist, has been warning for years about a bubble building up in the markets. In his latest missive, titled ‘Let The ... Read more »

Fed Launches Review of Possible Central Bank Digital Currency Officials solicit public comment, but are unlikely to decide soon whether to issue a government-backed cryptocurrency

“We look forward to engaging with the public, elected representatives, and a broad range of stakeholders as we examine the positives and negatives of a central bank digital currency in ... Read more »
On you will find lots of free English exam practice materials to help you improve your English skills: grammar, listening, reading, writing, ielts, toeic