Virtual Reality

First responder trainees don a wireless VR head-mounted display linked to a compatible desktop computer. Trainees see and hear autonomous, interactive victims who are programmed to simulate individuals with injuries consistent with an explosion in an underground space. Armed with a virtual medical kit, responders are tasked with triag ing and treating the victims on the scene. The VR environment can be made more challenging by increasing the environmental chaos, adding patients, or increasing the acuity of patient injuries.

The VR platform tracks and records their performance as they navigate the disaster scene. Output from the system provides feedback to participants on their performance. Eventually, we hope that the First Responder system will serve both as an effective replacement for expensive conventional training methods as well as a safe and efficient platform for research on current triage protocols.

The universal virtual patient is the cornerstone of the First Responder system. Individual patients can be completely customized with  respect to ethnicity, gender, and SALT category and types, severity, and numbers of injuries. In addition, patient attributes such as pulse, respiration, hearing, mood, position, and mobility can be configured. Scenarios are populated with virtual patients that are either selected by the trainee in the simulation or generated by the instructor using a universal patient configuration file. Scenes can be populated with as many as 32 patients. Each patient can be customized with up to 15 different injuries, including puncture wounds, amputations, lacerations, embedded shrapnel, tension pneumothorax, minor scrapes, respiratory distress, and hearing loss. 

Patients are programmed to continually check the status of their injuries. The assignment of specific injuries and their severity is linked to an appropriate response set for those injuries that then controls the patient’s behavior during the encounter, including vital signs (pulse and respiration), initial pose, ability to walk or respond to commands, and dialogue responses. In addition, the patient’s health can be programmed to decompensate during the time they are left unattended.

Patients are programmed to follow simple commands and respond to trainees’ questions with simple answers based on the types and severity of their injuries. User input is captured using Automated Speech Recognition (ASR) with the Windows Dictation Recognizer in Unity. Queries are matched to predefined phrase sets that can be easily modified to fine tune the patient’s vocabulary and ability to understand user input. Patients not only respond to key SALT triage protocol commands, such as “walk” or “wave,” but also will comply with other commands if their injuries allow, including “stand,” “sit,” “lie down,” or “move your arm/leg.” Patient voices are linked to a library

of audio files that were generated by voice actors to create authentic dialogue appropriate for interactions in a high-stress chaotic environment. Trainees are also able to take a patient’s pulse by receiving  haptic feedback through the system’s hand controllers. Other vital signs, such as respiration rate and airway assessment, are obtained through observation of the patient’s actions or vocalizations. If necessary, the trainee can reposition the patient to open his or her airway and receive auditory and visual feedback indicating the patient’s response.

Funded with a grant from the Agency for Healthcare Research and Quality 2019-2023

Department of Emergency Medicine, The Ohio State University College of Medicine, Columbus, Ohio, USA

Nicholas E. Kman MD

Center for Immersive Media, University of the Arts, Philadelphia, Pennsylvania, USA

Alan Price MFA

Advanced Computing Center for the Arts and Design, The Ohio State University, Columbus, Ohio, USA

Vita Berezina-Blackburn MA, MFA

Jeremy Patterson BFA

Alex Oliszewski MFA

Scott Swearingen MFA

Wexner Medical Center, James Cancer Hospital, Operations Improvement, The Ohio State University, Columbus, Ohio,

Kellen Maicher MFA

David P. Way MEd

Jillian McGrath MD

Ashish R. Panchal MD, PhD

Katherine Luu MD

Department of Obstetrics & Gynecology, The Ohio State University College of Medicine, Columbus, Ohio, USA

Douglas Danforth PhD

Attending physicians underwent a semi-structured interview about physician burnout and their

perceptions towards burnout. Participants were first asked to think about the ED environment

and how that environment contributes to workplace stress or burnout. Participants were then

taken to a large room whether they underwent an immersive high fidelity virtual reality

simulation of the emergency department (A side, B side and fast track) in which the subjects

work. The goal of the VR simulation was to provide the participants with visual and auditory

representations of the environment without the real-time distractions present in the actual

Emergency Department. The research personnel gave the subjects a brief tutorial on how to

utilize and navigate the VR simulation. They were asked to walk through the VR simulation and

provide commentary when they encountered areas that they associate with workplace stress.

The commentary was recorded on virtual sticky notes that were placed within the virtual

environment. Additionally, subjects filled out virtual comfort surveys for the three main attending

workstations.

Prior to the VR experience, participants described a general sense of “chaos” and distance to patients as contributing to burnout. When participants were immersed in the VR simulation, they tended to expand the relationship of the built environment to burnout by identifying more concrete contributors including layout, line of sight, noise, lighting, odors, distance, wayfinding, temperature and cleanliness.

A challenge to understanding burnout is physicians may tend to accept the environment as a fixed space and not see it as something they can control. Exploring the environmental space through the lens of burnout in an environment that stays constantly busy may prohibit introspection. VR has been increasingly utilized in healthcare for training purposes. Additionally VR has been shown to be beneficial in the perception of the built environment. The ability to capture a real space and create virtual instances might enable better introspection and discovery of environmental psychological stressors contributing to burnout.

Funded with a grant from the Emergency Medicine Foundation 2021-2022

PI: Jason Matthew Fields, MD; Associate Professor of Emergency Medicine & Research Director, Health Design Lab, Thomas Jefferson University

Co-PI: Alan Price, Director, Center for Immersive Media, University of the Arts

The VR experience incorporates digital models created from Ohio State’s Andean and Amazonian Cultural Artifact Collection

Using photogrammetry, select artifacts are reproduced at a high level of detail for close inspection in the VR environment and become a part of the interface and interaction design

User interaction includes the ability to play wind instruments. A wireless microphone is added to the Head Mount Display for sensing the force of the users breath as he or she blows towards virtual intruments.

As a concept-oriented, semi-directed environment, Sumac Puringashpa encourages users to experience and engage with Andean and Amazonian ways of looking at and conceiving of the world. Rather than explain or tell the user about Andean and Amazonian Cultures, the project aims to allow users to discover on their own, central concepts in Andean and Amazonian cultures such as:

• duality,

• reciprocity,

• transformation,

• mythic and mythical time-space, and recurring symbolic patterns

We achieve this by way of user-interactive interfaces such as musical instruments, a shaman’s seat of power (Amazonian bancu), and cultural artifacts that explore these concepts through attention to perspective, scale, directionality of gaze, and the interaction between visual and aural scapes.

Sumac Puringashpa showcases our unique approach to collaboration at ACCAD and the emphasis at Ohio State in studying the Andes and Amazonia as interconnected, rather than isolated regions.

The people in the town are autonomously driven characters wandering about and responding to each other as they pass. Initially, they appear to ignore the players presence, as if we are invisible to them. The men have a tendency to collect in groups on the street, engaging in conversation unknown to the players - with the intent to invoke paranoid curiosity – approaching a small group may cause them to disband.

Players must quell the growing conspiracies by distracting them – one means is by placing packages in their path, which they will gather and seek hiding places for the goods they have collected. As conspiracy groups grow in size, the player must exercise caution in approaching. Tossing a package into the group is one way to disband them, although the supply of packages is limited, and the player must deduce where more packages may be found. 

Players have other means of influencing the residents - including hypnosis – allowing the player to control the movement of a man. This can be used to infiltrate conspiracies, or to explore possibilities such as taking a hypnotized man on a bike ride. As a player freely explores the village, other networked players may eventually be discovered, with the realization that the player him or herself is also a sort of bathysphere with a single staring eye that has come to impose upon the people here.

Whether discovering the process independently or by observing other players, the game is not really a game in the common sense, but a commentary about entering and attempting to manipulate another culture while risking one's own discovery, with the impending capture and attempts by the residents to throw the player down a well. 

Electronics Alive, Scarfone/Hartley Gallery installation, University of Tampa, Jan 21-Feb 24, 2011
GlitchCon 2018, New Dimensions, Minneapolis–Saint Paul, Minnesota

Exhibited at
Faculty Exhibition, Visual Arts Gallery, University of Maryland Baltimore County,  Oct 1992