2024

1. 

   FLAME: A small language model for spreadsheet formulas

   Joshi, Harshit, Ebenezer, Abishai, Cambronero, José, Gulwani, Sumit, Kanade, Aditya, Le, Vu, Radiček, Ivan, and Verbruggen, Gust

   Proceedings of the AAAI Conference on Artificial Intelligence, 38 2024

   Abs HTML AAAI ’24

   The widespread use of spreadsheet environments by billions of users presents a unique opportunity for formula-authoring assistance. Although large language models, such as Codex, can assist in general-purpose languages, they are expensive to train and challenging to deploy due to their large model sizes (up to billions of parameters). Moreover, they require hundreds of gigabytes of training data. We present FLAME, a T5-based model trained on Excel formulas that leverages domain insights to achieve competitive performance with a substantially smaller model (60M parameters) and two orders of magnitude less training data. We curate a training dataset using sketch deduplication, introduce an Excel-specific formula tokenizer for our model, and use domain-specific versions of masked span prediction and noisy auto-encoding as pretraining objectives. We evaluate FLAME on formula repair, formula auto-completion, and a novel task called syntax reconstruction. FLAME (60M) can outperform much larger models, such as Codex-Davinci (175B), Codex-Cushman (12B), and CodeT5 (220M), in 6 out of 10 settings.

2023

1. 

   Repair Is Nearly Generation: Multilingual Program Repair with LLMs

   Joshi, Harshit, Cambronero, José, Gulwani, Sumit, Le, Vu, Radicek, Ivan, and Verbruggen, Gust

   Proceedings of the AAAI Conference on Artificial Intelligence, 37 2023

   Abs HTML AAAI ’23

   Most programmers make mistakes when writing code. Some of these mistakes are small and require few edits to the original program - a class of errors recently termed last mile mistakes. These errors break the flow for experienced developers and can stump novice programmers. Existing automated repair techniques targeting this class of errors are domain-specific and do not easily carry over to new domains. Transferring symbolic approaches requires substantial engineering and neural approaches require data and retraining. We introduce RING, a multilingual repair engine powered by a large language model trained on code (LLMC) such as Codex. Such a multilingual engine enables a flipped model for programming assistance, one where the programmer writes code and the AI assistance suggests fixes, compared to traditional code suggestion technology. Taking inspiration from the way programmers manually fix bugs, we show that a prompt-based strategy that conceptualizes repair as localization, transformation, and candidate ranking, can successfully repair programs in multiple domains with minimal effort. We present the first results for such a multilingual repair engine by evaluating on 6 different domains and comparing performance to domain-specific repair engines. We show that RING can outperform domain-specific repair engines in 3 of these domains. We also identify directions for future research using LLMCs for multilingual repair.

2022

1. 

   Neurosymbolic Repair for Low-Code Formula Languages

   Bavishi, Rohan*, Joshi, Harshit*, Sánchez, José Pablo Cambronero, Fariha, Anna, Gulwani, Sumit, Le, Vu, Radicek, Ivan, and Tiwari, Ashish

   In Proceedings of the ACM on Programming Languages (OOPSLA2) 2022

   Abs HTML OOPSLA ’22

   Most users of low-code platforms, such as Excel and PowerApps, write programs in domain-specific formula languages to carry out nontrivial tasks. Often users can write most of the program they want, but introduce small mistakes that yield broken formulas. These mistakes, which can be both syntactic and semantic, are hard for low-code users to identify and fix, even though they can be resolved with just a few edits. We formalize the problem of producing such edits as the last-mile repair problem. To address this problem, we developed LaMirage, a LAst-MIle RepAir-engine GEnerator that combines symbolic and neural techniques to perform last-mile repair in low-code formula languages. LaMirage takes a grammar and a set of domain-specific constraints/rules, which jointly approximate the target language, and uses these to generate a repair engine that can fix formulas in that language. To tackle the challenges of localizing the errors and ranking the candidate repairs, LaMirage leverages neural techniques, whereas it relies on symbolic methods to generate candidate repairs. This combination allows LaMirage to find repairs that satisfy the provided grammar and constraints, and then pick the most natural repair. We compare LaMirage to state-of-the-art neural and symbolic approaches on 400 real Excel and PowerFx formulas, where LaMirage outperforms all baselines. We release these benchmarks to encourage subsequent work in low-code domains.

2021

1. 

   Tweets Classification to Assist Human Moderation for Suicide Prevention

   Joshi, Harshit*, Sawhney, Ramit*, Nobles, Alicia*, and Shah, Rajiv Ratn

   In Proceedings of the International AAAI Conference on Web and Social Media 2021

   Abs HTML ICWSM ’21

   Social media platforms are already engaged in leveraging existing online socio-technical systems to employ just-in-time interventions for suicide prevention to the public. These efforts primarily rely on self-reports of potential self-harm content that is reviewed by moderators. Most recently, platforms have employed automated models to identify self-harm content, but acknowledge that these automated models still struggle to understand the nuance of human language (e.g., sarcasm). By explicitly focusing on Twitter posts that could easily be misidentified by a model as expressing suicidal intent (i.e., they contain similar phrases such as “wanting to die”), our work examines the temporal differences in historical expressions of general and emotional language prior to a clear expression of suicidal intent. Additionally, we analyze time-aware neural models that build on these language variants and factors in the historical, emotional spectrum of a user’s tweeting activity. The strongest model achieves high (statistically significant) performance (macro F1=0.804, recall=0.813) to identify social media indicative of suicidal intent. Using three use cases of tweets with phrases common to suicidal intent, we qualitatively analyze and interpret how such models decided if suicidal intent was present and discuss how these analyses may be used to alleviate the burden on human moderators within the known constraints of how moderation is performed (e.g., no access to the user’s timeline). Finally, we discuss the ethical implications of such data-driven models and inferences about suicidal intent from social media. Content warning: this article discusses self-harm and suicide.

2. 

   Suicide Ideation Detection via Social and Temporal User Representations using Hyperbolic Learning

   Joshi, Harshit*, Sawhney, Ramit*, Shah, Rajiv Ratn, and Flek, Lucie

   In Proceedings of the 2021 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, Volume 1 (Long and Short Papers) 2021

   Abs HTML Code NAACL-HLT ’21

   Recent psychological studies indicate that individuals exhibiting suicidal ideation increasingly turn to social media rather than mental health practitioners. Personally contextualizing the buildup of such ideation is critical for accurate identification of users at risk. In this work, we propose a framework jointly leveraging a user’s emotional history and social information from a user’s neighborhood in a network to contextualize the interpretation of the latest tweet of a user on Twitter. Reflecting upon the scale-free nature of social network relationships, we propose the use of Hyperbolic Graph Convolution Networks, in combination with the Hawkes process to learn the historical emotional spectrum of a user in a time-sensitive manner. Our system significantly outperforms state-of-the-art methods on this task, showing the benefits of both socially and personally contextualized representations.

3. 

   Robust Suicide Risk Assessment on Social Media via Deep Adversarial Learning

   Sawhney, Ramit, Joshi, Harshit, Gandhi, Saumya, Jin, Di, and Shah, Rajiv Ratn

   Journal of the American Medical Informatics Association 2021

   Abs HTML JAMIA ’21

   The prevalence of social media for sharing personal thoughts makes it a viable platform for the assessment of suicide risk. However, deep learning models are not able to capture the diverse nature of linguistic choices and temporal patterns that can be exhibited by a suicidal user on social media and end up overfitting on specific cues that are not generally applicable. We propose Adversarial Suicide assessment Hierarchical Attention (ASHA), a hierarchical attention model that employs adversarial learning for improving the generalization ability of the model.We assess the suicide risk of a social media user across 5 levels of increasing severity of risk. ASHA leverages a transformer-based architecture to learn the semantic nature of social media posts and a temporal attention-based long short-term memory architecture for the sequential modeling of a user’s historical posts. We dynamically generate adversarial examples by adding perturbations to actual examples that can simulate the stochasticity in historical posts, thereby making the model robust.Through extensive experiments, we establish the face-value of ASHA and show that it significantly outperforms existing baselines, with the F1 score of 64%. This is a 2% and a 4% increase over the ContextBERT and ContextCNN baselines, respectively. Finally, we discuss the practical applicability and ethical aspects of our work pertaining to ASHA, as a human-in-the-loop framework.Adversarial samples can be helpful in capturing the diverse nature of suicidal ideation. Through ASHA, we hope to form a component in a larger human-in-the-loop infrastructure for suicide risk assessment on social media.

4. 

   PHASE: Learning Emotional Phase-aware Representations for Suicide Ideation Detection on Social Media

   Joshi, Harshit*, Sawhney, Ramit*, Flek, Lucie, and Shah, Rajiv Ratn

   In Proceedings of 16th Conference of the European Chapter of the Association for Computational Linguistics 2021

   Abs HTML Code EACL ’21

   Recent psychological studies indicate that individuals exhibiting suicidal ideation increasingly turn to social media rather than mental health practitioners. Contextualizing the buildup of such ideation is critical for the identification of users at risk. In this work, we focus on identifying suicidal intent in tweets by augmenting linguistic models with emotional phases modeled from users’ historical context.We propose PHASE, a time-and phase-aware framework that adaptively learns features from a user’s historical emotional spectrum on Twitter for preliminary screening of suicidal risk. Building on clinical studies, PHASE learns phase-like progressions in users’ historical Plutchik-wheel-based emotions to contextualize suicidal intent. While outperforming state-of-the-art methods, we show the utility of temporal and phase-based emotional contextual cues for suicide ideation detection. We further discuss practical and ethical considerations.

5. 

   Towards Ordinal Suicide Ideation Detection on Social Media

   Sawhney, Ramit, Joshi, Harshit, Gandhi, Saumya, and Shah, Rajiv Ratn

   In Proceedings of 14th ACM International Conference On Web Search And Data Mining 2021

   Abs HTML Code WSDM ’21

   The rising ubiquity of social media presents a platform for individuals to express suicide ideation, instead of traditional, formal clinical settings. While neural methods for assessing suicide risk on social media have shown promise, a crippling limitation of existing solutions is that they ignore the inherent ordinal nature across fine-grain levels of suicide risk. To this end, we reformulate suicide risk assessment as an Ordinal Regression problem, over the Columbia-Suicide Severity Scale. We propose SISMO, a hierarchical attention model optimized to factor in the graded nature of increasing suicide risk levels, through soft probability distribution since not all wrong risk-levels are equally wrong. We establish the face value of SISMO for preliminary suicide risk assessment on real-world Reddit data annotated by clinical experts. We conclude by discussing the empirical, practical, and ethical considerations pertaining to SISMO ina larger picture, as a human-in-the-loop framework.

2020

1. 

   A Time-Aware Transformer Based Model for Suicide Ideation Detection on Social Media

   Sawhney, Ramit, Joshi, Harshit, Gandhi, Saumya, and Shah, Rajiv Ratn

   In Proceedings of the 2020 Conference on Empirical Methods in Natural Language Processing (EMNLP) 2020

   Abs HTML PDF Code Poster Slides EMNLP ’20

   Social media’s ubiquity fosters a space for users to exhibit suicidal thoughts outside of traditional clinical settings. Understanding the build-up of such ideation is critical for the identification of at-risk users and suicide prevention. Suicide ideation is often linked to a history of mental depression. The emotional spectrum of a user’s historical activity on social media can be indicative of their mental state over time. In this work, we focus on identifying suicidal intent in English tweets by augmenting linguistic models with historical context. We propose STATENet, a time-aware transformer based model for preliminary screening of suicidal risk on social media. STATENet outperforms competitive methods, demonstrating the utility of emotional and temporal contextual cues for suicide risk assessment. We discuss the empirical, qualitative, practical, and ethical aspects of STATENet for suicide ideation detection.