AI-supported writing technologies (AISWT) that provide grammatical suggestions, autocomplete sentences, or generate and rewrite text are now a regular feature integrated into many people’s workflows. However, little is known about how people perceive the suggestions these tools provide. In this paper, we investigate how Black American users perceive AISWT, motivated by prior findings in natural language processing that highlight how the underlying large language models can contain racial biases. Using interviews and observational user studies with 13 Black American users of AISWT, we found a strong tradeoff between the perceived benefits of using AISWT to enhance their writing style and feeling like “it wasn’t built for us”. Specifically, participants reported AISWT’s failure to recognize commonly used names and expressions in African American Vernacular English, experiencing its corrections as hurtful and alienating and fearing it might further minoritize their culture. We end with a reflection on the tension between AISWT that fail to include Black American culture and language, and AISWT that attempt to mimic it, with attention to accuracy, authenticity, and the production of social difference.

CCS Concepts: • Human-centered computing → Collaborative and social computing; Empirical studies in HCI; • Computing methodologies → Natural language processing. Additional Key Words and Phrases: Large Language Models, Bias in AI, African-American Vernacular English (AAVE), AI-Supported Writing Technologies (AISWT) ACM Reference Format: Jeffrey Basoah, Jay L. Cunningham, Erica Adams, Alisha Bose, Aditi Jain, Kaustubh Yadav, Zhengyang Yang, Katharina Reinecke, and Daniela Rosner. 2025. Should AI Mimic People? Understanding AI-Supported Writing Technology Among Black Users. Proc. ACM Hum.-Comput. Interact. 9, 7, Article CSCW242 (November 2025), 51 pages. https://doi.org/10.1145/3757423

Authors’ addresses: Jeffrey Basoah, jeffkb28@uw.edu, University of Washington, Seattle, Washington, USA; Jay L. Cunningham, jaylcham@uw.edu, University of Washington, Seattle, Washington, USA; Erica Adams, evadams6@uw.edu, University of Washington, Seattle, Washington, USA; Alisha Bose, abose04@uw.edu, University of Washington, Seattle, Washington, USA; Aditi Jain, ajain04@uw.edu, University of Washington, Seattle, Washington, USA; Kaustubh Yadav, kausty@uw.edu, University of Washington, Seattle, Washington, USA; Zhengyang Yang, yanzy@uw.edu, University of Washington, Seattle, Washington, USA; Katharina Reinecke, reinecke@cs.washington.edu, University of Washington, Seattle, Washington, USA; Daniela Rosner, dkrosner@uw.edu, University of Washington, Seattle, Washington, USA.

Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for components of this work owned by others than the author(s) must be honored. Abstracting with credit is permitted. To copy otherwise, or republish, to post on servers or to redistribute to lists, requires prior specific permission and/or a fee. Request permissions from permissions@acm.org. © 2025 Copyright held by the owner/author(s). Publication rights licensed to ACM. ACM 2573-0142/2025/11-ARTCSCW242 https://doi.org/10.1145/3757423

Introduction

Advances in natural language processing (NLP) are increasingly influencing many people’s lives by supporting their writing process. Basic word processors and other tools can now provide grammatical suggestions, autocomplete sentences, or even generate and rewrite text, as is the case for large language models (LLMs) like Open AI’s ChatGPT. While these AI-supported writing technologies (AISWT) have been hailed for revolutionizing the future of work [29], increasing productivity [18], and providing more equitable editing and writing help to a broad population [19, 67], Computer-Supported Cooperative Work and Social Computing (CSCW) researchers have repeatedly pointed out potential issues with the underlying LLMs [1, 10, 20, 25, 54]. For example, datasets and models used to train LLMs have been found to be more consistent with the values of Western and White people than with other groups of people [82]. Researchers have also discussed that databases and training data are often biased [41] and that the syntactic focus of NLP means that context and the use of language are all too often ignored by artificial intelligence (AI) [88]. What this means in practice is that LLMs commonly contain racial biases, including against African American Vernacular English (AAVE). Toxicity detection tools, for instance, are more likely to label expressions in AAVE as toxic than the equivalent expression in Standard American English (SAE) [42, 83, 97]. LLMs have been found to struggle in both generating and interpreting AAVE and generally performing better in generating SAE [26, 40]. While a notable body of work has examined biases in LLMs, studies that examine how individuals [16, 50, 74, 75, 77, 96], and particularly African American users [12, 22, 46, 69, 95], perceive their daily interactions with NLP tools have only just begun (see [4, 72]).

In this paper, we build on this growing body of CSCW and adjacent work by investigating how Black American users perceive AISWT. We pose the following research question: What are the expectations, apprehensions, and perceptions of Black American users regarding AI-supported writing technology? To answer this question, we employ a qualitative approach to understand the perceptions (gathered through semi-constructed virtual interviews) and experiences (observed in real-life context of a remote user study) of Black American users in their interactions with AISWT. Specifically, we examined the prior impressions and reactions of 13 Black American users to using AISWT as part of word processing software (Google Docs) and LLM (ChatGPT). We chose to focus on expectations, apprehensions, and perceptions because they represent key aspects of a user’s experience while engaging with technology [81, 99]. Examining Black American users’ expectations allows us to identify the baseline experience they anticipate when interacting with AISWT. Analyzing apprehensions sheds light on the barriers that deter Black American users from engaging with AISWT. Investigating perceptions enables us to uncover how Black American users understand and interpret AISWT. By addressing expectations, apprehensions, and perceptions, we aim to gain insight into the process of designing technologies in ways that emphasize not only functionality but also access, including tradeoffs revealed through this broadened engagement. Our study reveals the impact of AISWT on Black American users and their linguistic and cultural expressions. The findings underscore a prevailing sentiment among participants of a notable absence of consideration for Black individuals and groups in the development of AISWT, largely due to AISWT’s failure to recognize commonly used names and words within Black communities. Discomfort arises when AISWT attempts to replicate AAVE, with participants perceiving it as making unwarranted assumptions and casting doubt on the source of these assumptions. The study also sheds light on the perceived inefficiency of AISWT’s editing features and the technology’s potential impact on the perception of competence based on conformity to SAE. Despite these challenges, a substantial number of participants recognize the benefits of using AISWT to enhance their writing style and appear more professional, highlighting a mixed perspective on the technology’s utility.