MyPersonalCoach is similar to the MyBehavior project, with an important addon - MyBehavior suggestions are pushed in the right time and context to a smartwatch to make user's active.

The suggestions in the right moment promote two kinds of actions (i) repetition of same action to promote habit building (ii) novel or opportunisitic action to make user's active instantly.

I did this project for my 2014 internship at Intel. This project is currently inactive because of my lack of free time. I will expand this work in future. Here are a few slides on MyPersonalCoach link.

Recently in mobile and wearable computing, we have seen a rise in continuous sensing across different types of sensors and in device processing. Combination of these array of sensors and corresponding inference have capability to generate detailed non-trivial multidimensional information about the environment and the individual carrying the device. Also, there is a growing number of third-party applications that use these information for context-aware-computing, personal well-being, and just-in-time health care.

With SAINT, we propose a scalable sensing and real-time sensing and inference toolkit for mobile devices. SAINT uses a client-server architecture. The server can support clients with different sensing (e.g., accelerometer and location) and inference (e.g., human speech) streams on-demand. If more than one client ask for the same sensing or inference data then the server performs necessary computation (e.g., recognizing human speech) once and serves all the clients needing such data with inter process communication. Furthermore, internally SAINT server enables easy sharing of sensing and inference data among different recognition modules (e.g., sleep sensing can use movement and speech inference data) over a simple middle-man >bus like structure. This bus architecture makes any sensing or inference streams available for consumption for any other inference module. This sharing enables complex inference possible by reusing components that has been built before. We show that bus and client server architecture incur low overhead while enabling exponential increase in cpu time and power usage on the number of clients. Also, developers found SAINT framework easy to program and deploy.

Related Publications:

• Mashfiqui Rabbi, Thiago Caetano, Jean Costa, Saeed Abdullah, Mi Zhang, and Tanzeem Choudhury. SAINT: A Scalable Sensing and Inference Toolkit. Hotmobile 2015 poster

Bipolar disorder is a common form of mental illness. Medication is the current gold standard for bipolar depression treatment, but medication can have side effect. Social Rhythm Therapy (SRM) is a purely behavioral approach to reduce bipolar depression symptoms. SRM focuses on stabilizing routines to improve biploar symptoms. MoodRhythm is a mHealth version of SRM which facilitates stabilizing routine by monitoring daily routine behaviors and rewarding positive changes.

My role in this project was to infer daily behavior from sensor data.

Related Publications:

• Steven Voida, Mark Matthews, Saeed Abdullah, Mengxi C. Chi, Mattew Green, W. J. Jang, D. Hu, Jon Weinrich, P. Patil, Mashfiqui Rabbi, et al. Moodrhythm: tracking and supporting daily rhythms.In Proceedings of the 2013 ACM conference on Pervasive and ubiquitous computing adjunct publication, pages 67-70. ACM, 2013
• Stephen Voida, Tanzeem Choudhury, Geri Gay, Mark Matthews, Phil Adams, Mashfiqui Rabbi, JP Pollak, Mengxi Chi, Matthew Green, Andrew Campbell, Nic Lane, and Hong Lu. Personal Informatics Can Be Stressful: Collecting, Reflecting, and Embedding Stress Data in Personal Informatics. To Appear in the Proceeding of Personal Informatics Workshop, CHI.

Neural signals are everywhere just like mobile phones. We propose to use neural signals to control mobile phones for hands-free, silent and effortless human-mobile interaction. Until recently, devices for detecting neural signals have been costly, bulky and fragile. We present the design, implementation and evaluation of the NeuroPhone system, which allows neural signals to drive mobile phone applications on the iPhone using cheap off-the-shelf wireless electroencephalography (EEG) headsets. We demonstrate a brain-controlled address book dialing app, which works on similar principles to P300-speller brain-computer interfaces: the phone flashes a sequence of photos of contacts from the address book and a P300 brain potential is elicited when the flashed photo matches the person whom the user wishes to dial. EEG signals from the headset are transmitted wirelessly to an iPhone, which natively runs a lightweight classifier to discriminate P300 signals from noise. When a person’s contactphoto triggers a P300, his/her phone number is automatically dialed. NeuroPhone breaks new ground as a brainmobile phone interface for ubiquitous pervasive computing. We discuss the challenges in making our initial prototype more practical, robust, and reliable as part of our on-going research.

Related Publications:

• Andrew T. Campbell, Tanzeem Choudhury, Shaohan Hu, Hong Lu, Matthew Mukerjee, Mashfiqui Rabbi, and Rajeev Raizada. NeuroPhone: Brain-Mobile Phone Interface using a Wireless EEG Headset. Appears in the Proceedings of MobiHeld 2010 [pdf]
  

StressSense is a continuous sensing application that captures audio from phone microphones and classifies the audio into stressful or regular conversations. The basic intuition is that, during stressful episodes, the airflow through the vocal cord changes (e.g., pitch, speaking rate, teager energy) and this airflow dynamics can be modeled and classified.

We have an implementation of StressSense on the Android platform, so please let us know if you are interested to try StressSense (mrabbi@fas.harvard.edu).

Related publication
Hong Lu, Mashfiqui Rabbi, Gokul Chittaranjan, Denise Frauendorfer, Marianne Schmidt, Andrew Campbell, Daneil Gatica-Perez, and Tanzeem Choudhury. StressSense: Detecting Stress in Unconstrained Acoustic Environments using Smartphones. Appears in the Proceedings of Ubicomp 2012, September 2012.

This 2011 project captured fine grained motion and privacy-sensitive audio data from Intel MSP, and derived different metrics that correlated with commonly used surveys for assessing mental well-being.

This project later inspired the BeWell and the StudentLife project.

Related Publications:

• Mashfiqui Rabbi, Shahid Ali, Tanzeem Choudhury, and Ethan Berke. Passive and In-situ Assessment of Mental and Physical Well-being using Mobile Sensors. To appear in the Proceedings of Ubicomp 2011, September 2011. Beijing, China.
• Ethan Berke, Tanzeem Choudhury, Shahid Ali, and Mashfiqui Rabbi. Objective Sensing of Activity and Sociability: Mobile Sensing in the Community. Appears in the Annals of Family Medicine, Volume 9, Issue 4, Pages 344-350, July 2011. [commentary]

A growing body of research has examined the capacity for smartphone sensors to identify human behavioral markers related to mental health. GPS features developed at our Center for Behavioral Intervention Technologies (CBITs) have become the standard sensed geographic state features used in research. These GPS features, which measure geographic state, include: 1) time spent at home, 2) number of locations visited, 3) variability in locations visited, 4) variability in time spent across locations (entropy), 5) circadian movement, 6) time spent in transitions between locations, and 7) distance traveled in transitions. Research using these features, summarized in a recent systematic review, found these features are related to depression across many studies.

People have control over how much time they spend at home, the numbers and types of places they go, and how long they spend in those places. This is a rare case where a new measurement tool uncovers a novel potential mental health treatment target. Smartphones are an excellent medium through which to intervene on geographic state, as they measure GPS data objectively and continuously. Most digital mental health interventions to date attempt to translate treatment strategies developed outside of digital environments (e.g., cognitive restructuring or behavioral activation) to digital platforms. This would mark the first psychological treatment target that is not translated, but rather is digital from inception in its measurement and intervention.

Adolescents and young adults (AYAs) with cancer demonstrate suboptimal oral chemotherapy adherence, increasing their risk of cancer relapse. It is unclear how everyday time-varying contextual factors (eg, mood) affect their adherence, stalling the development of personalized mobile health (mHealth) interventions. Poor engagement is also a challenge across mHealth trials; an effective adherence intervention must be engaging to promote uptake. This research aims to determine the temporal associations between daily contextual factors and 6-mercaptopurine (6-MP) adherence and explore the proximal impact of various engagement strategies on ecological momentary assessment survey completion.

At the Children's Hospital of Philadelphia, AYAs with acute lymphoblastic leukemia or lymphoma who are prescribed prolonged maintenance chemotherapy that includes daily oral 6-MP are eligible, along with their matched caregivers. Participants used an ecological momentary assessment app called ADAPTS (Adherence Assessments and Personalized Timely Support)—a version of an open-source version of SARA app that was modified for AYAs with cancer through a user-centered process—and complete surveys in bursts over 6 months. Theory-informed engagement strategies will be microrandomized to estimate the causal effects on proximal survey completion.

Participants were randomized to receive inspirational quotes, memes, and altruistic messages at 0.5 probability. We found memes increased next-day self-report by 51% on weekends and inspirational quotes increased the same day self- report by 8% when self-report was completed the day prior.

Repeated spikes in blood glucose levels can exacerbate conditions for type-2 diabetes mellitus (T2DM) and develop T2DM for pre-diabetics. Glucose spikes are defined as rapid increases in blood glucose levels. These spikes slowly get back to baseline for T2DM and can develop insulin resistance for pre-diabetics. Research shows that even two minutes of low-intensity walking within 15-45 minutes of the start of the spike can curb the spike and reduce its harmful effects. Given that T2DM is one of the most expensive diseases to manage, just-in-time walking intervention at high risk moments of glucose spikes can have significant public health impact.

Our present work takes several meaningful steps to address this problem. For the first time in extant literature, we define and predict early indicators of glucose spikes from continuous glucose monitors and develop a testbed for developing a just-in-time walking interventionto that uses these early indicators. We first establish several risk definitions associated with glucose ``spikes" after talking to domain experts that are actionable for just-in-time interventions. Then, we implement inception-based models that leverage data from continuous glucose monitors to predict any impending risk event and how many such events may happen rest of the day. These models are trained using a self-supervised approach involving the clustering of individuals to enhance performance. Finally, we adopt Sequential Risk Time Estimation (seqRTS), a sampling-based algorithm that integrates all risk predictions to create a testbed to learn just-in-time interventions. These interventions are scheduled based on user availability and preferences and are designed to enhance adherence without overwhelming the user with excessive notifications. Through simulations with real-world data, we reflect on how user's blood glucose levels change. We validate our approach using two datasets comprising glucose monitoring data: `ShanghaiT2DM', a public dataset involving 109 individuals with T2DM, featuring an average of 14+/-2.1 days of CGM data, and a private dataset we compiled from 3,000 T2DM users, encompassing 30+/-4.1 days of CGM data per user. Our findings demonstrate the potential of our proposed methods in managing T2DM through tailored, user-centric interventions.

150 minutes of weekly moderate to vigorous activity (MVPA) can reduce all-cause mortality risk by 33%. Yet more than 50% US population do less than 150 MVPA minutes weekly. Wearables can capture fine-grained MVPA data and insights based on this data can increase MVPA. Yet, most of these data-driven insights are reside inside apps, and most people do not open health apps on their own. Push notifications can surface these personalized data-driven insights without require users to open the app and thus increasing their MVPA.

We created pi2-messages that create personalized data-driven insights from a user's MVPA history and delivers those insights inside push notifications multiple times a day. pi2-messages uses three types of insights to make persuasive push messages: closing a steak, staying consistent with the user's usual activity, and reminding on daily MVPA goal progress. During the day, participants are randomized six times daily, starting from 6 AM every three hours. 6 AM morning notification motivates participants about their daily MVPA goal. 9 AM, 12 PM, 3 PM, and 6 PM notifications try to increase MVPA within 1-3 hours. Finally, the 9 PM notification asked the user to plan MVPA for the next day. These notifications motivated users to complete a streak, stay on track to their daily goal, or stay consistent with their usual activity within a specific hour of the day.

Twenty-two participants enrolled in a pilot micro-randomized. 9 AM, 12 PM, 3 PM, and 6 PM push notifications increased MVPA by 18% within 75 minutes post-intervention (p=0.077, 95% CI = 0.98 to 1.43). Messages that reminded about daily goal progress were more effective at increasing MVPA (21%, p=0.056, 95% CI = 0.99 to 1.46) than messages that asked users to stay consistent with their usual activity. We observed no longer-term negative effects of these interventions. 6 AM and 9 PM planning messages had a null effect on increasing MVPA within 24 hours post-intervention. Participants in the exit survey mentioned that push interventions with personalized insights are helpful reminders. However, some participants reported the notifications as status updates and wanted them to be more motivational.

The sub-goals app divides a daily goal into several sub-goals where each subgoal is assigned to a specific time segment of the day. When all sub-goals are added together, they sum to the daily goal. These sub-goals also reduce the burden as they ask users to be active when they are usually active. The time horizon to complete each subgoal is sooner than the daily goal, thus users will likely discount those rewards less and be more motivated. In iOS17, Apple shipped a feature similar to sub-goals in their Fitness+ app.

MyBehavior learns a user's physical activity and dietary behavior from phone data and suggests personalized changes to these behaviors. MyBehavior is the first app that combined mobile data and reinforcement learning to provide personalized health recommendations.

See our JMIR and Ubicomp 2015 papers for more details on MyBehavior. We recently extended MyBehavior for chronic back pain and published an article in JMIR.

Besides passive sensing, ecological momentary assessments (EMAs) are one of the primary methods to collect in-the-moment data in ubiquitous computing and mobile health. While EMAs have the advantage of low recall bias, a disadvantage is that they frequently interrupt the user and thus long-term adherence is generally poor. In this paper, we propose a less-disruptive self-reporting method, “assisted recall,” in which in the evening individuals are asked to answer questions concerning a moment from earlier in the day assisted by contextual information such as location, physical activity, and ambient sounds collected around the moment to be recalled. Such contextual information is automatically collected from phone sensor data, so that self-reporting does not require devices other than a smartphone. We hypothesized that providing assistance based on such automatically collected contextual information would increase recall accuracy (i.e., if recall responses for a moment match the EMA responses at the same moment) as compared to no assistance, and we hypothesized that the overall completion rate of evening recalls (assisted or not) would be higher than for in-the-moment EMAs. We conducted a two-week study (N=54) where participants completed recalls and EMAs each day. We found that providing assistance via contextual information increased recall accuracy by 5.6% (p = 0.032) and the overall recall completion rate was on average 27.8% (p < 0.001) higher than that of EMAs.

Related Publications:

• Mashfiqui Rabbi, Katherine Li, H Yanna Yan, Kelly Hall, Predrag Klasnja, Susan Murphy ReVibe: A Context-assisted Evening Recall Approach to Improve Self-report Adherence Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies, 3, 4, 1--27 (2019), ACM New York, NY, USA:[pdf]

BeWell is a continuous sensing application that infers physical activity, sleep and social interaction from mobile sensor data. BeWell then aggregates the number of minutes of physical activity, social interaction, and sleep in a day, and populates a virtual aquarium in the background of the phone.

BeWell was a milestone project that tied multi-modal sensing with health feedback. It kickstarted a flurry of follow-up projects that include MyBehavior and the StudentLife project at Dartmouth.

Related Publications:

• Nicholas D. Lane, Mu Lin, Mashfiqui Rabbi, Xiaochao Yang, Hong Lu, Giuseppe Cardone, Shahid Ali, Ethan Berke, Andrew T. Campbell, Tanzeem Choudhury Bewell: Sensing sleep, physical activities and social interactions to promote wellbeing. Mobile Networks and Applications 19, no. 3 (2014): 345-359.
• Mu Lin, Nicholas Lane, Mashfiqui Rabbi, Xiaochao Yang, Hong Lu, Giuseppe Cardone, Shahid Ali, Afsaneh Doryab, Ethan Berke, Andrew Campbell, and Tanzeem Choudhury. BeWell+: Multi-dimensional Wellbeing Monitoring with Community-guided User Feedback and Energy Optimization Appears in the Proceedings of Wireless Health 2012, October 2012.
• Nicholas D. Lane, Mashfiqui Rabbi, Mu Lin, Xiaochao Yang, Afsaneh Doryab, Hong Lu, Shahid Ali, Tanzeem Choudhury, Andrew Campbell, and Ethan Berke, BeWell: A Smartphone Application to Monitor, Model and Promote Wellbeing, Pervasive Health 2011-- 5th International ICST Conference on Pervasive Computing Technologies for Healthcare, Dublin, 23-26 May 2011