AI Talk: Metaverse and health care

April 8, 2022 / By V. “Juggy” Jagannathan, PhD

Last month I saw an excellent article in Time Magazine by Sasha Brodsky about how virtual reality is changing health care. The thrust of the article was that virtual reality is impacting telemedicine, surgery and medical training. Given this backdrop, I thought it would be instructive to explore what is happening in the virtual reality world, which now goes by the popular moniker “metaverse.”

The term “metaverse” is not really new. It was coined in 1992 by science fiction writer Neal Stephenson to denote 3D virtual space. Facebook recently changed their name to Meta to announce to the world that they are going to be a serious player in metaverse, aka virtual world. In addition to the usual suspects in big tech, this site lists tons of startups jumping into the metaverse fray. I came across this recent encyclopedic entry providing a detailed explanation of the concept. Here is a short excerpt of their definition of metaverse:

“The Metaverse is the post-reality universe, a perpetual and persistent multiuser environment merging physical reality with digital virtuality. It is based on the convergence of technologies that enable multisensory interactions with virtual environments, digital objects and people such as virtual reality (VR) and augmented reality (AR).”

One of the keys in the above definition is “convergence of technologies.” What technologies are we talking about? Well, here are a few key ones: artificial intelligence (AI), Internet of Things (IoT), 5G and, beyond networking, virtual and augmented reality headsets and blockchains. Why blockchains? It’s one of the driving forces behind multi-user-gaming aimed at sharing data securely.

One term that keeps cropping up in the metaverse sphere is avatar. It is a Sanskrit word that alludes to God taking on a human role. Of course, we are also familiar with the blockbuster movie from James Cameron, Avatar, which popularized this term. Avatar in the metaverse is just a digital version of the users interacting with the system. There are even companies that will custom craft an avatar to look just like you. Metaverse is big on creating social environments (as evidenced by Facebook becoming Meta) and multi-user gaming systems. If one looks beyond the entertainment and social interaction sphere, the biggest opportunity for this technology lies in the education and training world.

Immersive environments can provide an engaging world for all manner of training. Pilots get their training in extremely expensive realistic simulation environments. The VR environments now provide an inexpensive and realistic alternative for such training. Be it surgery or dental implants, the brave new world of training surgeons and dentists can now be shaped by VR software.

The big advantage this modality of training provides is a seamless transition from training to reality. In training the sensory input and body location involved in practicing, say surgery, is simulated. But this can transition to the real world where the simulation is replaced by sensory input from actual patients, an AR scenario! Now, if you have your VR headsets on and controlling some end effectors supported by AR, and network connectivity is 5G, then there is no requirement that the patient be next to you! They can be hundreds or thousands of miles away – leading to a telepresence scenario. And that is telemedicine.

The technology is still nascent and evolving. End effector robots, in the case of surgery, need to be more responsive. Then there is the whole physical side of wearing the headsets. Leaving aside the problems associated with too much time spent on social platforms, currently it is uncomfortable to wear these headsets and goggles for long stretches of time. People get dizzy and disoriented. Another problem is content. Training simulated environments is non-trivial and such content is not easy to build. This article refers to the VR world as “Waze for surgeons,” helping them navigate the human body with surgical instruments, just like Google maps helps one navigate the real world.

But the future is bright for the health care use cases for metaverse. One can see why Sasha Brodsky focused on telemedicine, surgery and training in her article on virtual reality – the technology, once it matures, is an excellent fit.

I am always looking for feedback and if you would like me to cover a story, please let me know! Leave me a comment below or ask a question on my blogger profile page.

“Juggy” Jagannathan, PhD, is Director of Research for 3M M*Modal and is an AI Evangelist with four decades of experience in AI and Computer Science research.