How Companies Scour Our Digital Lives for Clues to Our Health
An emerging field, digital phenotyping, tries to assess people’s well-being based
on their interactions with digital devices.
By NATASHA SINGERFEB. 25, 2018
Your digital footprint — how often you post on social media, how quickly you scroll through your contacts, how frequently you check your phone late at night — could hold clues to your physical and mental health.
That at least is the theory behind an emerging field, digital phenotyping, that is trying to assess people’s well-being based on their interactions with digital devices. Researchers and technology companies are tracking users’ social media posts, calls, scrolls and clicks in search of behavior changes that could correlate with disease symptoms. Some of these services are opt-in. At least one is not.
People typically touch their phones 2,617 per day, according to one study — leaving a particularly enticing trail of data to mine.
“Our interactions with the digital world could actually unlock secrets of disease,†said Dr. Sachin H. Jain, chief executive of CareMore Health, a health system, who has helped study Twitter posts for signs of sleep problems. Similar approaches, he said, might someday help gauge whether patients’ medicines are working.
“It could help with understanding the effectiveness of treatments,†he said.
The field is so new and so little studied, however, that even proponents warn that some digital phenotyping may be no better at detecting health problems than a crystal ball.
If a sociable person suddenly stopped texting friends, for instance, it might indicate that he or she had become depressed, said Dr. Steve Steinhubl, director of digital medicine at the Scripps Translational Science Institute in San Diego. Or “it could mean that somebody’s just going on a camping trip and has changed their normal behavior,†he said.
“It’s this whole new potential for snake oil,†Dr. Steinhubl said.
That is not stopping the rush into the field — by start-ups and giants like Facebook — despite questions about efficacy and data privacy.
Scanning for Suicidal Thoughts
One of the most ambitious efforts is being conducted by Facebook.
The company recently announced that it was using artificial intelligence to scan posts and live video streams on its social network for signs of possible suicidal thoughts. If the system detects certain language patterns — such as friends posting comments like “Can I help?†or “Are you O.K.?†— it may assign a certain algorithmic score to the post and alert a Facebook review team.
In some cases, Facebook sends users a supportive notice with suggestions like “Call a helpline.†In urgent cases, Facebook has worked with local authorities to dispatch help to the user’s location. The company said that, over a month, its response team had worked with emergency workers more than 100 times.
Some health researchers applauded Facebook’s effort, which wades into the complex and fraught realm of mental health, as well intentioned. But they also raised concerns. For one thing, Facebook has not published a study of the system’s accuracy and potential risks, such as inadvertently increasing user distress.
“It’s a great idea and a huge unmet need,†Dr. Steinhubl said. Even so, he added, Facebook is “certainly right up to that line of practicing medicine not only without a license, but maybe without proof that what they are doing provides more benefit that harm.â€
For another thing, Facebook is scanning user posts in the United States and some other countries for signs of possible suicidal thoughts without giving users a choice of opting out of the scans.
“Once you are characterized as suicidal, is that forever associated with your name?†said Frank Pasquale, a law professor at the University of Maryland who studies emerging health technologies. “Who has access to that information?â€
Will Nevius, a Facebook spokesman, said Facebook deleted the algorithmic scores associated with posts after 30 days. The cases involving emergency responders are kept in a separate system that is not tied to users’ profiles, he said.
Facebook said it had worked with suicide prevention groups when developing the effort. Mr. Nevius added that publishing a useful study would be complex because of the difficulty in removing personal data and “the delicate nature of the posts.â€
Detecting Depression in Clicks
Therapists traditionally diagnose depression by observing patients and asking them how they feel. Mindstrong Health, a mental health start-up in Palo Alto, Calif., is observing people’s smartphone use.
The company has developed a research platform to continuously monitor users’ phone habits, looking at changes in taps and clicks for hints about mood and memory changes associated with depression.
“We are building digital smoke alarms for people with mental illness,†said Dr. Thomas R. Insel, a Mindstrong co-founder and a former director of the National Institute of Mental Health.
Mindstrong’s research app tracks 1,000 smartphone-related data points — like how long it takes someone to scroll through a contact list and click on a name. The start-up recruited 200 volunteers to participate in pilot studies. Dr. Insel said a few of the signals, like changes in users’ keyboard accuracy and speed, correlated with similar motor skills changes that researchers could measure in lab tests.
Now the company is participating in a large government-funded study of trauma patients. Part of it involves using the Mindstrong platform to study whether patients who go on to develop post-traumatic stress disorder also develop corresponding changes in their smartphone use.
“We’ve got these really interesting statistical signals with very high correlations,†Dr. Insel said. “But whether that’s going to work in the real world of clinical care is something we’re looking at right now.â€
He added that Mindstrong had tapped law and ethics experts to help examine the implications of its technology and develop ethical frameworks for using it.
“You want to think through all the unintended consequences early on,†Dr. Insel said, “so they don’t come back to bite you.â€
Vetting Calls for Signs of Stress
The traditional use of a phone — talking — is also being examined for health clues. Sharecare, a digital health company based in Atlanta, offers a wellness app with an optional feature that analyzes users’ stress levels during phone calls.
The system uses pattern recognition technology to categorize users’ speech, the company said. After each call, the system delivers reports like “you seemed anxious†or “you seemed balanced.†It also characterizes users’ relationships with the people they call in terms of attitudes like “dominance†or “affection.â€
Jeff Arnold, a co-founder of Sharecare, described the voice scan as “an emotional selfie.â€
“If I can tell you your stress level in real time, it will in itself change your behavior,†said Mr. Arnold, who previously founded WebMD.
(Could not bring Tweet/image over - see link.)
Health insurers and self-insured companies use Sharecare to promote wellness and manage health care costs. Sharecare is working with the Georgia Institute of Technology to study the effectiveness of its voice analysis service.
Jiten Chhabra, a health tech researcher at the university’s Interactive Media Technology Center, said volunteers who tried the voice-scanning feature reported feeling less stressed afterward. But he said it was too soon to tell whether the stress analysis itself directly caused the change — or whether volunteers had simply become more relaxed in their daily lives.
The company does not record the content of the calls it scans, it said. But the app did collect phone numbers for people on the other side of calls from Sharecare users, according to an analysis by The New York Times. The service did not inform people on the phone with Sharecare users that their relationships were being analyzed.
Jennifer Martin Hall, a spokeswoman for Sharecare, said the way the company protected data “makes it practically impossible for any Sharecare employees to access a phone number in the call information.†She added that characterizing users’ voice “analysis in terms of ‘relationships’ helps contextualize the relevance of their stress and enables them to be more mindful day to day.â€
(Could not bring image over - see link.)
Sharecare says its voice analysis feature does not record users’ phone calls, but it sends a notice after every call analyzing Sharecare users’ relationships — without informing people on the other end of the call that the app is doing so.
Other researchers said such pervasive scanning could also have the opposite effect — increasing stress on otherwise healthy people.
“It’s like we’re in school forever,†Professor Pasquale said, “and we’re being graded in all these ways forever by all the companies that have the most data about us.â€
An emerging field, digital phenotyping, tries to assess people’s well-being based
on their interactions with digital devices.
By NATASHA SINGERFEB. 25, 2018
Your digital footprint — how often you post on social media, how quickly you scroll through your contacts, how frequently you check your phone late at night — could hold clues to your physical and mental health.
That at least is the theory behind an emerging field, digital phenotyping, that is trying to assess people’s well-being based on their interactions with digital devices. Researchers and technology companies are tracking users’ social media posts, calls, scrolls and clicks in search of behavior changes that could correlate with disease symptoms. Some of these services are opt-in. At least one is not.
People typically touch their phones 2,617 per day, according to one study — leaving a particularly enticing trail of data to mine.
“Our interactions with the digital world could actually unlock secrets of disease,†said Dr. Sachin H. Jain, chief executive of CareMore Health, a health system, who has helped study Twitter posts for signs of sleep problems. Similar approaches, he said, might someday help gauge whether patients’ medicines are working.
“It could help with understanding the effectiveness of treatments,†he said.
The field is so new and so little studied, however, that even proponents warn that some digital phenotyping may be no better at detecting health problems than a crystal ball.
If a sociable person suddenly stopped texting friends, for instance, it might indicate that he or she had become depressed, said Dr. Steve Steinhubl, director of digital medicine at the Scripps Translational Science Institute in San Diego. Or “it could mean that somebody’s just going on a camping trip and has changed their normal behavior,†he said.
“It’s this whole new potential for snake oil,†Dr. Steinhubl said.
That is not stopping the rush into the field — by start-ups and giants like Facebook — despite questions about efficacy and data privacy.
Scanning for Suicidal Thoughts
One of the most ambitious efforts is being conducted by Facebook.
The company recently announced that it was using artificial intelligence to scan posts and live video streams on its social network for signs of possible suicidal thoughts. If the system detects certain language patterns — such as friends posting comments like “Can I help?†or “Are you O.K.?†— it may assign a certain algorithmic score to the post and alert a Facebook review team.
In some cases, Facebook sends users a supportive notice with suggestions like “Call a helpline.†In urgent cases, Facebook has worked with local authorities to dispatch help to the user’s location. The company said that, over a month, its response team had worked with emergency workers more than 100 times.
Some health researchers applauded Facebook’s effort, which wades into the complex and fraught realm of mental health, as well intentioned. But they also raised concerns. For one thing, Facebook has not published a study of the system’s accuracy and potential risks, such as inadvertently increasing user distress.
“It’s a great idea and a huge unmet need,†Dr. Steinhubl said. Even so, he added, Facebook is “certainly right up to that line of practicing medicine not only without a license, but maybe without proof that what they are doing provides more benefit that harm.â€
For another thing, Facebook is scanning user posts in the United States and some other countries for signs of possible suicidal thoughts without giving users a choice of opting out of the scans.
“Once you are characterized as suicidal, is that forever associated with your name?†said Frank Pasquale, a law professor at the University of Maryland who studies emerging health technologies. “Who has access to that information?â€
Will Nevius, a Facebook spokesman, said Facebook deleted the algorithmic scores associated with posts after 30 days. The cases involving emergency responders are kept in a separate system that is not tied to users’ profiles, he said.
Facebook said it had worked with suicide prevention groups when developing the effort. Mr. Nevius added that publishing a useful study would be complex because of the difficulty in removing personal data and “the delicate nature of the posts.â€
Detecting Depression in Clicks
Therapists traditionally diagnose depression by observing patients and asking them how they feel. Mindstrong Health, a mental health start-up in Palo Alto, Calif., is observing people’s smartphone use.
The company has developed a research platform to continuously monitor users’ phone habits, looking at changes in taps and clicks for hints about mood and memory changes associated with depression.
“We are building digital smoke alarms for people with mental illness,†said Dr. Thomas R. Insel, a Mindstrong co-founder and a former director of the National Institute of Mental Health.
Mindstrong’s research app tracks 1,000 smartphone-related data points — like how long it takes someone to scroll through a contact list and click on a name. The start-up recruited 200 volunteers to participate in pilot studies. Dr. Insel said a few of the signals, like changes in users’ keyboard accuracy and speed, correlated with similar motor skills changes that researchers could measure in lab tests.
Now the company is participating in a large government-funded study of trauma patients. Part of it involves using the Mindstrong platform to study whether patients who go on to develop post-traumatic stress disorder also develop corresponding changes in their smartphone use.
“We’ve got these really interesting statistical signals with very high correlations,†Dr. Insel said. “But whether that’s going to work in the real world of clinical care is something we’re looking at right now.â€
He added that Mindstrong had tapped law and ethics experts to help examine the implications of its technology and develop ethical frameworks for using it.
“You want to think through all the unintended consequences early on,†Dr. Insel said, “so they don’t come back to bite you.â€
Vetting Calls for Signs of Stress
The traditional use of a phone — talking — is also being examined for health clues. Sharecare, a digital health company based in Atlanta, offers a wellness app with an optional feature that analyzes users’ stress levels during phone calls.
The system uses pattern recognition technology to categorize users’ speech, the company said. After each call, the system delivers reports like “you seemed anxious†or “you seemed balanced.†It also characterizes users’ relationships with the people they call in terms of attitudes like “dominance†or “affection.â€
Jeff Arnold, a co-founder of Sharecare, described the voice scan as “an emotional selfie.â€
“If I can tell you your stress level in real time, it will in itself change your behavior,†said Mr. Arnold, who previously founded WebMD.
(Could not bring Tweet/image over - see link.)
Health insurers and self-insured companies use Sharecare to promote wellness and manage health care costs. Sharecare is working with the Georgia Institute of Technology to study the effectiveness of its voice analysis service.
Jiten Chhabra, a health tech researcher at the university’s Interactive Media Technology Center, said volunteers who tried the voice-scanning feature reported feeling less stressed afterward. But he said it was too soon to tell whether the stress analysis itself directly caused the change — or whether volunteers had simply become more relaxed in their daily lives.
The company does not record the content of the calls it scans, it said. But the app did collect phone numbers for people on the other side of calls from Sharecare users, according to an analysis by The New York Times. The service did not inform people on the phone with Sharecare users that their relationships were being analyzed.
Jennifer Martin Hall, a spokeswoman for Sharecare, said the way the company protected data “makes it practically impossible for any Sharecare employees to access a phone number in the call information.†She added that characterizing users’ voice “analysis in terms of ‘relationships’ helps contextualize the relevance of their stress and enables them to be more mindful day to day.â€
(Could not bring image over - see link.)
Sharecare says its voice analysis feature does not record users’ phone calls, but it sends a notice after every call analyzing Sharecare users’ relationships — without informing people on the other end of the call that the app is doing so.
Other researchers said such pervasive scanning could also have the opposite effect — increasing stress on otherwise healthy people.
“It’s like we’re in school forever,†Professor Pasquale said, “and we’re being graded in all these ways forever by all the companies that have the most data about us.â€
We seem to be losing sight of the knowledge that human life is fraught with emotional ups-and-downs that are usually situational (although those situations can last months or years sometimes). What seems unbearable at 17 is intensely irritating at 27 and trivial at 37. It may not hit your radar after that.
Distraught phone calls are a function of some female/female relationships and that can last for years. It doesn't mean the caller is going to kill herself - she's venting to someone who isn't going to hold it against her later. I'm the go-to recipient for a lot of calls like that because I'm agreeable and calm. I just listen and ask vague questions.
Introverts (like me) probably show very high stress levels on phone calls from stupid or intrusive people. It doesn't mean we're going to cut ourselves or buy black tar heroin. It means we don't like to be interrupted.
It's normal to be worried, sad, angry, anxious, or confused at various times. Being sad or bored is not life-threatening depression. Being angry is not a one-way ticket to interpersonal violence. Being 'nervous' in social situations is not the same thing as an 'anxiety disorder'. Worrying about actual events or likely probable future events is not a mental illness - it's correct or anticipation (also correct). It's how you solve problems that you can identify.
I'm a fairly normal person despite having OCD but the reality is that my life sucks just the same way it sucks for anybody. People die, I get sad. People are stupid, I get irritated. Things happen, I worry. Sometimes I call my girlfriends and just vent. If I was on social media, maybe that would be data but it wouldn't tell you anything about my state of mind in a clinical sense.
NYT
Comment