Personalized Health Care

Using technology to track trends in your health

Published Feb. 3, 2017

Fitness trackers and smart watches are just two examples of wearable products on the market that are helping people better monitor their daily activity and providing data such as their heart rate. RIT plans to leverage its strengths in new media design and behavioral, social, and data sciences to enhance personalized health care and access through various intelligent, mobile, and wearable technologies.

The following feature story is one of five signature interdisciplinary research areas RIT is current exploring through advanced research. Each of these research areas is reflective of a growing significance in society, and all are endeavors where RIT can make high-impact contributions in the fields.

Personal Health Care Technology

Providing evidence that mobile and wearable technologies could predict trends in people’s health and prevent disease is a priority for the National Institutes of Health as part of its strategic plan. The NIH’s goal closely aligns with one of RIT’s signature research initiatives in personalized health care technology. RIT will invest $1 million in this area over the next five years.

A doctor and nurse show a patient her heart rate on a tablet app

Unobtrusively Monitoring Patient’s Heart Rate: Technology using video cameras and software could unobtrusively monitor a person’s cardiac activity. The technology is currently in development at RIT. Called video photoplethysmography (VPG), it would have the potential to monitor a person’s vital signs while they waited in a hospital emergency department to be seen by a doctor. Software would analyze the VPG signal to detect any changes in the person’s heart rate or an indication of an arrhythmia based on subtle changes in the color of the patient’s skin. The doctor would be alerted if the patient’s condition started to deteriorate.

RIT plans to leverage its resources and expertise from a wide number of research centers and faculty researchers to facilitate further innovation in health care and build upon its history in applying science and technology to support individuals with a wide range of challenges. For example, in health care delivery, communication between doctors and their patients can often be a huge hurdle especially for certain populations like the deaf and hard of hearing, the elderly, and people with varying physical and cognitive abilities.

“Directly monitoring someone greatly facilitates assessing their health status,” said Dan Phillips, Faculty Associate and Lead of RIT’s Partnership for Access Technology Research and Development. “If you connect a measurement device to somebody, objective data will be collected that can be directly provided to a clinical professional for evaluation, regardless of the individuals ability to communicate. This is incredibly important for individuals in which communication abilities are limited or non-existent.”

Gill Tsouri, RIT associate professor of electrical and microelectronic engineering, is working on developing technology to unobtrusively monitor a person’s cardiac activity. It’s called video photoplethysmography (VPG) and it monitors vital signs using a video camera and signal processing algorithms.

For someone waiting to be treated in a hospital emergency department, their status can be monitored using videos cameras and software that would immediately alert staff if the patient’s condition starts to deteriorate. This technology could also be used in a home setting. Software analyzes the video signals to detect arrhythmias and heart rates. The technology is currently being tested in clinical trials in collaboration with J.P. Couderc of the Heart Research Follow-Up Program at the University of Rochester Medical Center.

Here’s how the technology works: Light reflecting off the skin exhibits subtle changes in color as the heart delivers blood to and from the face. Tracking color variations over time provides a signal comparable to one obtained using a photoplethysmography skin sensor.

Another example of technology that ties in with personalized health care for heart patients is an industry-funded sensor system integrated into a toilet seat. Currently under development, it could revolutionize the way in which people are monitored and treated after being hospitalized for heart failure. The technology is the brainchild of David Borkholder, Bausch and Lomb Professor of Microsystems at RIT; Nick Conn, RIT Ph.D. candidate in the microsystems engineering program; and Dr. Karl Schwarz, director of the Echocardiography Laboratory at the University of Rochester. The seat measures a patient’s vital signs such as heart rate, heart rhythm, blood pressure, arterial oxygen saturation, and weight.

Toilet seat with sensors and it's corresponding app

Predicting Health Trends for Heart Failure Patients: A toilet seat embedded with sensors is currently under development by David Borkholder, Bausch and Lomb Professor of Micro-systems at RIT, and Dr. Karl Schwarz, director of UR’s Echocardiography Laboratory. The seat (above left), designed for patients who have suffered from heart failure, can produce a patient’s vital signs from skin contact with the seat. The patient’s data could then be displayed on a user interface for both the doctor and patient to review (above right). User interface design credit: RIT Visiting Assistant Professor Hye-Jin Nae

Borkholder says that by the patient just sitting on the toilet seat doctors are able to obtain reliable measurements that require skin contact, such as the electrocardiogram.

“You get skin contact from the person without them doing anything out of the ordinary,” says Borkholder. “You can obtain the patient’s data at the same time every day, for example, when they first get up in the morning to use the bathroom. The person at this point hasn’t had any food or caffeine. So all of the issues that confound the data when a person goes to the doctor’s office are no longer part of the equation. You are collecting data at the same time on a regular basis which is critical for trend analysis.”

Borkholder and Schwarz are currently working with hospitalized patients at the UR and are also conducting a six-month, in-home trial with heart failure patients. The consistent data collection over time can show if there are trends that could predict deteriorating health conditions. This would allow doctors to intervene early enough so that the patient does not have to be readmitted to the hospital.

From Data Collection to Design Thinking

Once the data is collected from these various technology applications, the challenge then becomes how to effectively present all of this information to the user. This is where RIT’s new media design program, led by Adam Smith, will take the technology and customize a digital interface for the client to view the data. The interface could be viewed on someone’s phone, a tablet, or a touch screen at the doctor’s office.

“We bring to the table the design approaches and philosophies that focus on the user through visual language,” says Smith. “You always have to be thinking about the needs and wants of the consumer. We figure out what that data needs to look like for the both the patient and the doctor and can solve the information problem so that the data is usable.”

Facilitating Better Communication: A collaborative student team project (Team Mendo) created by RIT new media developers and new media designers explored how mobile, handheld apps and large touchscreens could facilitate better communication between doctors and patients.

The program is built on engaging multidisciplinary teams of undergraduate students, made up of new media interactive development majors and new media design majors. Each year, teams complete a senior capstone project for an industry partner. Past clients have included Adobe, Xerox, Rock and Roll Hall of Fame, and the UR’s HIV Trials Unit. Last year, a student team created a digital touchscreen to display a patient’s prescriptions, diet, and exercise recommendations to be reviewed with the doctor in the office. The interface could also be viewed on the patient’s mobile devices.

Digital Therapies Impacting Positive Behavioral Health

RIT is also using medical interactive technologies like cell phone apps, virtual tools, and avatars to change people’s behaviors to be more positive and healthy. Caroline Easton, an RIT professor of forensic clinical psychology, specializes in working with people who have issues with substance dependency and family conflict. Easton and Richard Doolittl, vice dean of RIT's College of Health Sciences and Technology, lead a multidisciplinary team of faculty and students developing interactive digital therapies.

Al-Virt® is an avatar who engages clients in virtual-role playing exercises and model-conflict skills in between their therapy sessions. Initially designed four years ago by Alan Gesek, ’11, ’14 (illustration, medical illustration), Al-Virt® recently underwent a makeover after clients provided feedback about his appearance and emotions. Clayton Scavone, an RIT game design student, created a more realistic looking avatar.

“The avatar helps the clients apply their coping skills,” says Easton. “For example, the offenders may yell and swear and become agitated with their partner and other people in their environment. Through our digital therapies, we teach them healthier ways to be assertive in their communication skills as opposed to using aggressive behaviors.”

Digital Behavioral Health Coach: Al-Virt® is a digital behavioral health coach developed at RIT to teach healthy communication and conflict resolution skills to offenders undergoing treatment for substance abuse and aggressive behavior.

Al-Virt® rewards the clients for trying to take a step in the right direction regarding improving their own behaviorial health. Easton says that research shows that practicing a changed healthier behavior leads to better treatment outcomes such as a reduction in substance abuse and aggression.

Data-Driven Decision Making

Collecting data of people’s use of language and social cues on social media platforms such as Twitter, Facebook, and Instagram is giving RIT researchers a glimpse at their behavior and health risks. RIT computer science professor Christopher Homan studies computational social network analysis. Homan says that due to the vast amount of information that people post from their personal mobile devices in any given day, researchers can see how people’s behavior on social media could impact their health.

“Does the person’s language on social media suggest they are happy, sad, or depressed?” asked Homan. “If someone is unhappy, they may develop unhealthy behaviors like eating unhealthy foods, failing to exercise, or taking drugs,” said Homan. “By understanding behavior, you understand the personalized health risks that certain people have and also what kinds of treatments they are amenable to.”