publications

2024

1. IMWUT

   Playlogue: Dataset and Benchmarks for Analyzing Adult-Child Conversations During Play

   Kalanadhabhatta, Manasa, Rastikerdar, Mohammad Mehdi, Rahman, Tauhidur, Grabell, Adam,  and Ganesan, Deepak

   Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. Nov 2024

   Abstract DOI PDF Dataset

   There has been growing interest in developing ubiquitous technologies to analyze adult-child speech in naturalistic settings such as free play in order to support children’s social and academic development, language acquisition, and parent-child interactions. However, these technologies often rely on off-the-shelf speech processing tools that have not been evaluated on child speech or child-directed adult speech, whose unique characteristics might result in significant performance gaps when using models trained on adult speech. This work introduces the Playlogue dataset containing over 33 hours of long-form, naturalistic, play-based adult-child conversations from three different corpora of preschool-aged children. Playlogue enables researchers to train and evaluate speaker diarization and automatic speech recognition models on child-centered speech. We demonstrate the lack of generalizability of existing state-of-the-art models when evaluated on Playlogue, and show how fine-tuning models on adult-child speech mitigates the performance gap to some extent but still leaves considerable room for improvement. We further annotate over 5 hours of the Playlogue dataset with 8668 validated adult and child speech act labels, which can be used to train and evaluate models to provide clinically relevant feedback on parent-child interactions. We investigate the performance of state-of-the-art language models at automatically predicting these speech act labels, achieving significant accuracy with simple chain-of-thought prompting or minimal fine-tuning. We use inhome pilot data to validate the generalizability of models trained on Playlogue, demonstrating its utility in improving speech and language technologies for child-centered conversations. The Playlogue dataset is available for download at https://huggingface.co/datasets/playlogue/playlogue-v1.

2. CHI

   Multi-stakeholder Perspectives on Mental Health Screening Tools for Children

   Kalanadhabhatta, Manasa, Mateo Santana, Adrelys, Mayorga, Lynnea, Rahman, Tauhidur, Ganesan, Deepak,  and Grabell, Adam

   In Proceedings of the 2024 CHI Conference on Human Factors in Computing Systems. May 2024

   Abstract DOI PDF

   Pediatric mental health is a growing concern around the world, affecting children’s social-emotional development and increasing the risk of poor behavioral outcomes later in life. However, obtaining a behavioral diagnosis in early childhood is challenging due to lack of access to resources, low parental mental health literacy, and children’s dependence on several stakeholders to coordinate care for them. While app-based, at-home screening tools could offer a scalable and convenient diagnostic solution for families, stakeholder perspectives on their utility and usability remain to be examined. This work reports on a survey of child mental health practitioners and interviews with parents to illustrate existing barriers to care that stakeholders encounter, the perceived benefits of app-based screening tools in meeting their needs, and the challenges in scaling up these tools. We identify where stakeholders agree or disagree, delineate key design tensions, and provide recommendations for the development of future screening technologies.

2023

1. UbiComp Adjunct

   Towards Accurate and Scalable Mental Health Screening Technologies for Young Children

   Kalanadhabhatta, Manasa, Ganesan, Deepak,  and Rahman, Tauhidur

   In Adjunct Proceedings of the 2023 ACM International Joint Conference on Pervasive and Ubiquitous Computing & the 2023 ACM International Symposium on Wearable Computing. Oct 2023

   Abstract DOI PDF

   Mental, emotional, and behavioral disorders are highly prevalent in preschool-aged children and can significantly affect their social-emotional development and adaptive functioning. However, identifying signs of problematic behavior at this age is extremely challenging due to several structural and phenomenological barriers. This work leverages mobile and wearable devices to build accurate, usable, and scalable assessment tools that can be deployed in home settings to screen for common disorders in young children. It describes the development of novel screening algorithms that utilize behavioral and neurophysiological signals recorded during brief, naturalistic tasks, and presents stakeholder perspectives toward the usability and clinical utility of such screening tools.

2. ACII

   Detecting PTSD Using Neural and Physiological Signals: Recommendations from a Pilot Study

   Kalanadhabhatta, Manasa, Roy, Shaily, Grant, Trevor, Salekin, Asif, Rahman, Tauhidur,  and Bergen-Cico, Dessa

   In 11th International Conference on Affective Computing and Intelligent Interaction (ACII). Sep 2023

   Abstract DOI PDF

   Post-traumatic stress disorder (PTSD) is a serious condition that is characterized by negative mood and affect, hyperarousal, irritability, and reactivity, as well as deterioration of cognitive processes such as attention and memory. Timely identification and treatment of PTSD symptoms can significantly improve symptom management and recovery. However, accurate prediction of PTSD outside clinical settings is often challenging. In this work, we investigate whether deficits in cognitive performance can be used to classify individuals with and without PTSD. We further examine whether neural and physiological signals such as prefrontal cortex activity, heart rate, respiration, and electrodermal activity recorded in conjunction with cognitive task performance can be leveraged to improve PTSD classification. Our results indicate that working memory tasks can achieve an F1 score of 0.80 at classifying individuals with PTSD, which can be further improved to 0.91 by combining multimodal information from neurophysiological signals. Based on our findings, we provide recommendations for in-the-wild PTSD classification.

2022

1. ACII

   Extracting Multimodal Embeddings via Supervised Contrastive Learning for Psychological Screening

   Kalanadhabhatta, Manasa, Mateo Santana, Adrelys, Ganesan, Deepak, Rahman, Tauhidur,  and Grabell, Adam

   In 10th International Conference on Affective Computing and Intelligent Interaction (ACII). Oct 2022

   Abstract DOI PDF Slides

   The diagnosis of psychological disorders in early childhood is of utmost importance given their severe impact on children’s academic and social skills as well as general adaptive functioning. Wearable and video-based systems have the potential to collect important diagnostic information in the form of neurophysiological and behavioral signals. However, accurate prediction of psychological disorder status from multimodal data streams necessitates their combination into meaningful features for classification models. In this work, we present a multitask supervised contrastive learning approach to learn useful multimodal embeddings from functional Near-Infrared Spectroscopy, galvanic skin response, and facial video data collected during a frustration-inducing task. The generated embeddings are able to accurately infer emotion regulation-related psychological disorders with an F1 score of 0.91, having significant implications for early-childhood mental health diagnoses.

2. IMWUT

   EarlyScreen: Multi-scale Instance Fusion for Predicting Neural Activation and Psychopathology in Preschool Children

   Kalanadhabhatta, Manasa, Mateo Santana, Adrelys, Zhang, Zhongyang, Ganesan, Deepak, Grabell, Adam S,  and Rahman, Tauhidur

   Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies. Jul 2022

   Abstract DOI PDF Slides

   Emotion dysregulation in early childhood is known to be associated with a higher risk of several psychopathological conditions, such as ADHD and mood and anxiety disorders. In developmental neuroscience research, emotion dysregulation is characterized by low neural activation in the prefrontal cortex during frustration. In this work, we report on an exploratory study with 94 participants aged 3.5 to 5 years, investigating whether behavioral measures automatically extracted from facial videos can predict frustration-related neural activation and differentiate between low- and high-risk individuals. We propose a novel multi-scale instance fusion framework to develop EarlyScreen - a set of classifiers trained on behavioral markers during emotion regulation. Our model successfully predicts activation levels in the prefrontal cortex with an area under the receiver operating characteristic (ROC) curve of 0.85, which is on par with widely-used clinical assessment tools. Further, we classify clinical and non-clinical subjects based on their psychopathological risk with an area under the ROC curve of 0.80. Our model’s predictions are consistent with standardized psychometric assessment scales, supporting its applicability as a screening procedure for emotion regulation-related psychopathological disorders. To the best of our knowledge, EarlyScreen is the first work to use automatically extracted behavioral features to characterize both neural activity and the diagnostic status of emotion regulation-related disorders in young children. We present insights from mental health professionals supporting the utility of EarlyScreen and discuss considerations for its subsequent deployment.

2021

1. PervasiveHealth

   FatigueSet: A Multi-modal Dataset for Modeling Mental Fatigue and Fatigability

   Kalanadhabhatta, Manasa, Min, Chulhong, Montanari, Alessandro,  and Kawsar, Fahim

   In 15th International Conference on Pervasive Computing Technologies for Healthcare (PervasiveHealth). Dec 2021

   Abstract DOI PDF Dataset

   A comprehensive understanding of fatigue and its impact on performance is a prerequisite for fatigue management systems in the real world. However, fatigue is a multidimensional construct that is often poorly defined, and most prior work does not take into consideration how different types of fatigue collectively influence performance. The physiological markers associated with different types of fatigue are also underexplored, hindering the development of fatigue management technologies that can leverage mobile and wearable sensors to predict fatigue. In this work, we present FatigueSet, a multi-modal dataset including sensor data from four wearable devices that are collected while participants are engaged in physically and mentally demanding tasks. We describe the study design that enables us to investigate the effect of physical activity on mental fatigue under various situations. FatigueSet facilitates further research towards a deeper understanding of fatigue and the development of diverse fatigue-aware applications.

2. JMIR

   Effect of Sleep and Biobehavioral Patterns on Multidimensional Cognitive Performance: Longitudinal, In-the-Wild Study

   Kalanadhabhatta, Manasa, Rahman, Tauhidur,  and Ganesan, Deepak

   J Med Internet Res. Feb 2021

   Abstract DOI PDF

   Background: With nearly 20% of the US adult population using fitness trackers, there is an increasing focus on how physiological data from these devices can provide actionable insights about workplace performance. However, in-the-wild studies that understand how these metrics correlate with cognitive performance measures across a diverse population are lacking, and claims made by device manufacturers are vague. While there has been extensive research leading to a variety of theories on how physiological measures affect cognitive performance, virtually all such studies have been conducted in highly controlled settings and their validity in the real world is poorly understood. Objective: We seek to bridge this gap by evaluating prevailing theories on the effects of a variety of sleep, activity, and heart rate parameters on cognitive performance against data collected in real-world settings. Methods: We used a Fitbit Charge 3 and a smartphone app to collect different physiological and neurobehavioral task data, respectively, as part of our 6-week-long in-the-wild study. We collected data from 24 participants across multiple population groups (shift workers, regular workers, and graduate students) on different performance measures (vigilant attention and cognitive throughput). Simultaneously, we used a fitness tracker to unobtrusively obtain physiological measures that could influence these performance measures, including over 900 nights of sleep and over 1 million minutes of heart rate and physical activity metrics. We performed a repeated measures correlation (rrm) analysis to investigate which sleep and physiological markers show association with each performance measure. We also report how our findings relate to existing theories and previous observations from controlled studies. Results: Daytime alertness was found to be significantly correlated with total sleep duration on the previous night (rrm=0.17, P<.001) as well as the duration of rapid eye movement (rrm=0.12, P<.001) and light sleep (rrm=0.15, P<.001). Cognitive throughput, by contrast, was not found to be significantly correlated with sleep duration but with sleep timing—a circadian phase shift toward a later sleep time corresponded with lower cognitive throughput on the following day (rrm=–0.13, P<.001). Both measures show circadian variations, but only alertness showed a decline (rrm=–0.1, P<.001) as a result of homeostatic pressure. Both heart rate and physical activity correlate positively with alertness as well as cognitive throughput. Conclusions: Our findings reveal that there are significant differences in terms of which sleep-related physiological metrics influence each of the 2 performance measures. This makes the case for more targeted in-the-wild studies investigating how physiological measures from self-tracking data influence, or can be used to predict, specific aspects of cognitive performance.

2018

1. CHI EA

   Analysis of Peer Group Behavior Among University Students

   Kunchay, Sahiti,  and Kalanadhabhatta, Lakshmi Manasa

   In Extended Abstracts of the 2018 CHI Conference on Human Factors in Computing Systems. Apr 2018

   Abstract DOI PDF

   Satisfactory peer group interactions within a university, through the formation of close associations, define a student’s personality and help in deterring the rise of depression caused by academic, financial or emotional troubles. In this work, we conduct a pre-study survey of 177 students in a University setting to assess the requirement for a smartphone-based study to detect and monitor group formation, evolution and engagement. The preliminary results from this investigation reveal that students social interactions are not limited to one but several groups, and the satisfaction levels associated with each type of group are indicative of the average time spent engaging with said group(s). Intra-group bond strength took precedence as a satisfaction determinant over the location or activity engaged in. Further, we present design recommendations for a minimally invasive smartphone-based study.

2017

1. SenSys Adjunct

   Smartphone-based Qualitative Analyses of Social Activities During Family Time

   Sahiti, Kunchay,  Kalanadhabhatta, Lakshmi Manasa, Bhunia, Suman Sankar, Singhal, Akshit,  and Majethia, Rahul

   In Proceedings of the First International Workshop on Human-centered Sensing, Networking, and Systems. Nov 2017

   Abstract DOI PDF

   With the evolution of nuclear families and diverse career options, families as social groups are spending lesser time together than in the past decades. In this work, we study both quantitative as well as qualitative aspects of time spent with family members through a smartphone-based pervasive study on a sample of 12 families over 14 days. Further, we also examine the perception of 78 millennials on what they feel about, and expect from, the time they spend with their families, however long it may be. We aim to identify the key parameters that shape family life in this day and age, along with examining the participation of individuals of various roles within the family in activities such as conversations, workout sessions, eating together and other social interactions. Among all activities detected to be performed by families reporting high satisfaction with familial life, Eating Together and Using Smartphones Together emerged as the most prominent ones. We discover a greater disparity among the habits of family members, especially millennials, staying away from each other as compared to those staying together.

2. UbiComp Adjunct

   Application Overchoice: Preliminary Lessons from a Longitudinal Study

   Kalanadhabhatta, Lakshmi Manasa, Mathur, Akhil, Majethia, Rahul,  and Kawsar, Fahim

   In Proceedings of the 2017 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2017 ACM International Symposium on Wearable Computers. Sep 2017

   Abstract DOI PDF

   In this paper, we investigate how smartphone users navigate the dilemma of application overchoice, i.e., the scenario of having multiple competing apps available to serve a similar purpose. We analyze app installs, app usage behavior and notification attendance behavior to paint an initial picture of app overchoice and to explore how overchoice is impacted by smartphone notifications. We hope that this paper will provoke discussions and more research in the UbiTtention community on developing systems that help users navigate the dilemma of overchoice.

3. IMWUT

   Moving Beyond Market Research: Demystifying Smartphone User Behavior in India

   Mathur, Akhil,  Kalanadhabhatta, Lakshmi Manasa, Majethia, Rahul,  and Kawsar, Fahim

   Proceedings of the ACM on Interactive, Mobile, Wearable, and Ubiquitous Technologies. Sep 2017

   Abstract DOI PDF

   Large-scale mobile data studies can reveal valuable insights into user behavior, which in turn can assist system designers to create better user experiences. After a careful review of existing mobile data literature, we found that there have been no large-scale studies to understand smartphone usage behavior in India – the second-largest and fastest growing smartphone market in the world. With the goal of understanding various facets of smartphone usage in India, we conducted a mixed-method longitudinal data collection study through an Android app released on Google Play. Our app was installed by 215 users, and logged 11.9 million data points from them over a period of 8 months. We analyzed this rich dataset along the lines of four broad facets of smartphone behavior – how users use different apps, interact with notihcations, react to different contexts, and charge their smartphones – to paint a holistic picture of smartphone usage behavior of Indian users. This quantitative analysis was complemented by a survey with 55 users and semi-structured interviews with 26 users to deeply understand their smartphone usage behavior. While our first-of-its-kind study uncovered many interesting facts about Indian smartphone users, we also found striking differences in usage behavior compared to past studies in other geographical contexts. We observed that Indian users spend significant time with their smartphones after midnight, continuously check notifications without attending to them and are extremely conscious about their smartphones’ battery. Perhaps the most dramatic finding is the nature of mobile consumerism of Indian users as shown by our results. Taken together, these and the rest of our findings demonstrate the unique characteristics that are shaping the smartphone usage behavior of Indian users.

2016

1. MobiSys Adjunct

   AnnoTainted: Automating Physical Activity Ground Truth Collection Using Smartphones

   Majethia, Rahul, Singhal, Akshit,  Kalanadhabhatta, Lakshmi Manasa, Sahiti, Kunchay, Kishore, Shubhangi,  and Nandwani, Vijay

   In Proceedings of the 3rd International on Workshop on Physical Analytics. Jun 2016

   Abstract DOI PDF

   In this work, we provide motivation for a zero-effort crowdsensing task: auto-annotated ground truth collection for physical activity recognition. Data obtained through Smartphones for classification of human activities is prone to discrepancies, which reiterates the need for better and larger activity datasets. Artificial data generation algorithms fail to efficiently generate quality instances for minority data. In the proposed model, crowd-sourced sensor data is classified by a robust classifier built by researchers ground up. We nominate a Generic Classifier with ≥ 95% accuracy for this purpose. Data collection and distribution models which ensure that the crowd client receives non-skewed, quality data from locations with higher degree of activity occurrence are elucidated upon. Also integrated within our proposed model are Location-Specific Classifiers, which can be utilized by developers to optimize on location-specific tasks. Effective validation of classified activities using diverse sensor data streams improves the proposed classifier systems and boosts ground-truth accuracy.

2. COMSNETS

   PeopleSave: Recommending Effective Drugs Through Web Crowdsourcing

   Majethia, Rahul, Mishra, Varun, Singhal, Akshit,  Kalanadhabhatta, Lakshmi Manasa, Sahiti, Kunchay,  and Nandwani, Vijay

   In 2016 8th International Conference on Communication Systems and Networks (COMSNETS). Jan 2016

   Abstract DOI PDF

   In this paper, we describe PeopleSave - a drug recommendation and feedback system for doctors on the basis of contextual patient reviews crowd-sourced from the Internet. Unlike other systems proposed in the past, we filter information sources to check for crowdsourcing feasibility and then assess the drug’s effectiveness based on its reported detrimental effect on a patient. This helps in eliminating certain drugs that would almost certainly have an adverse effect on the patient’s health and thereby obtain a set of recommendable drugs. These recommendations are further refined by analyzing the sentiment behind the opinions of patients who have been administered these drugs in the past. The resultant set of prescribable drugs agrees with those suggested by the consulted physicians for the considered sample set of diabetes patients. The critical assessment of the prototype system for Diabetes Type II drugs by both doctors and patients also reiterates the need for a feedback system that can possibly go a long way in improving patient experience of a drug. This leads us to conclude that PeopleSave, as a combination of the recommendation system prototype and the proposed feedback system, can be successful in improving the process of prescription of medicines for a varied range of medical conditions.