The Promise and the Gap: Rethinking Accessibility in Conversational AI

For most of the history of consumer technology, interacting with a device meant one extra step for him that it didn’t mean for me. Screen readers translating visual layouts into audio. Workarounds for image-heavy interfaces. The constant friction of technology designed first for sighted users, then retrofitted for everyone else. It was never impossible, but it was always a little more effortful, a little more burdensome, a little more dependent on whether whoever built the thing had bothered to think about him.

Then something changed.

Recently, my dad found a recipe online that was saved as an image. A PNG, no text, totally inaccessible to a screen reader. He uploaded it to ChatGPT and asked it to describe what it said. And it did. He got the recipe. He cooked the thing. A piece of technology did something genuinely useful for him that no one explicitly designed it to do. It wasn’t a feature that shipped with an accessibility label. It was just a byproduct of a system that can look at an image and reason about what’s in it.

That’s remarkable. That’s actually worth celebrating.

That story contains almost everything I want to say about where conversational AI is and where it needs to go.

The interface is the accommodation

There’s a reason conversational interfaces feel, at first glance, like a genuine accessibility win. No menus to navigate. No buttons to find. No visual hierarchy to decode. Just language, the most human thing we have. For someone who is blind, a well-designed conversational AI doesn’t require a screen reader workaround or an accessibility mode buried three settings deep. The interface is the accommodation. For someone who is Deaf and communicates primarily through written language, a text-based chat interface may actually be more natural than voice-first tools ever were. For someone with motor impairments who struggles with a mouse or touchscreen, typing or dictating a request to an AI can replace entire workflows that used to require precise physical interaction.

This is meaningful. Traditional UI accessibility has historically been retrofitted, an afterthought bolted onto interfaces designed primarily for sighted, hearing, able-bodied users. Conversational AI inverts that, at least partially. The baseline interaction model is language, and language is one of the most universal human channels we have.

The blank text box is not neutral

Consider someone with a significant cognitive disability, a traumatic brain injury, severe ADHD, or an intellectual disability. Conversational AI can be harder for these users, not easier. The open-ended prompt, that blank text box, that “How can I help you?”, places enormous cognitive load on the user. Traditional UI, for all its flaws, provides scaffolding: buttons name options, menus constrain choices, visual affordances suggest what actions are possible. A chat interface offers none of that. It demands that a user already know what they want, how to articulate it, and how to evaluate a long-form response. That’s a high bar.

Then there are people with speech and language disorders: stuttering, aphasia, dysarthria, selective mutism. Voice-based conversational AI can be actively hostile to these users. Systems trained primarily on fluent, standard speech patterns may fail to recognize them, misinterpret them, or worst of all, confidently return the wrong answer and erode trust. The promise of “just talk to it” assumes a kind of speech that a significant portion of the population simply doesn’t produce.

And this isn’t just about disability. The bias baked into early speech recognition systems reflects something uglier: who was in the room when those systems were built, and whose voice was treated as the default. Research by Koenecke et al. found that error rates for Black American speakers were nearly double those of white American speakers across commercial ASR systems, a disparity that existed across every vendor tested, driven largely by the underrepresentation of Black speakers in training data. Researchers have since argued that when errors in transcribing certain voices are systematic and persistent, they move beyond technical limitation into something more like moral disregard, a signal that some linguistic identities are less worthy of accurate recognition. “Just talk to it” was never neutral. It was a design choice that reflected whose voice the system was built to hear.

What about users with AAC devices, augmentative and alternative communication tools used by people with ALS, cerebral palsy, autism, and other conditions? These users often communicate at a different pace, with different syntax, and through different input modalities. Many conversational AI systems haven’t been meaningfully tested with AAC users at all.

And then there’s literacy. Conversational AI, in its current dominant form, is fundamentally text- or speech-dependent. For users with low literacy, or whose primary language is not well-represented in training data, the interface that feels frictionless to one user can feel opaque and alienating to another.

“Verify the output” assumes the user can

Let’s go back to my dad and those four recipe translations.

In the field of ethical AI, we talk a lot about transparency. One of our most consistent recommendations is that AI-generated content should be labeled as such, and that users should be encouraged to verify outputs for accuracy. It’s good guidance. It’s responsible guidance.

But it contains an assumption so embedded that we rarely name it: that the user can verify.

My dad couldn’t. He had no way to look at the original image and check the AI’s work against it. He was entirely dependent on the system being correct, and when four systems gave him four different answers, he had no independent means to arbitrate between them. He had to call me. He had to introduce a human intermediary to do the thing the technology had promised to do.

This is a real and underexamined problem. We build AI systems that produce confident, fluent, well-formatted output, and we tell users to verify it. That’s a reasonable ask when your user has the ability to verify. It is not a reasonable ask when the thing the user is trying to accomplish is accessing information they otherwise can’t access. In those cases, the AI isn’t a tool that helps someone do a thing. It’s the only mechanism by which they can do the thing at all. Accuracy isn’t a nice-to-have. It’s the whole point.

Acknowledgment is not the same as practice

Honestly? Not consistently.

There is real work happening. The University of Illinois is collaborating with Microsoft, Google, Amazon, and several nonprofits on the Speech Accessibility Project, an initiative specifically aimed at making voice recognition useful for people with atypical speech. Google’s ongoing Parrotron project is designed to help users with atypical speech be understood by both other people and speech interfaces. And a 2024 systematic review of five years of AI accessibility research identified a critical gap in work addressing speech and cognitive impairments, which is at least an acknowledgment that the gap exists.

But the gap between acknowledgment and practice is wide. A 2024 survey found that 63% of disabled people either know nothing about AI or have only heard about it in passing, and just 9% actively use it. That’s a striking finding for a technology routinely described as inherently more accessible. Accessibility testing in AI development tends to focus on output quality for mainstream users, with disability treated as a compliance checkbox rather than a design input.

There’s also the compounding problem: when AI systems fail users with disabilities, those users are less likely to be in the feedback loops that improve future models. The data flywheel that makes AI better over time can systematically exclude the people who most need better design. We’ve seen this before. The racial disparities in speech recognition weren’t an accident. They were the predictable result of building systems without the people most affected by them.

Not just accessible by default. Adaptively accessible.

Conversational AI has something traditional UI never had: it’s malleable in a way that doesn’t require redesigning a component library. The same underlying system can, in principle, support radically different interaction patterns based on user need. That’s the real opportunity, not just “accessible by default,” but adaptively accessible.

A few directions worth pursuing seriously:

Part of the way there is not a finish line

Conversational AI is, on balance, a step forward for accessibility. My dad cooking a recipe he couldn’t have read six years ago is evidence of that. That’s real.

But “part of the way there” is not a finish line. And the places where the technology falls short don’t fall evenly across all users. They fall hardest on the people who were already carrying the most weight, who had the fewest workarounds, the least margin for error, the greatest need for the system to simply work.

The blank text box is not neutral. The assumption of fluent speech is not neutral. A confident, fluent, plausibly wrong answer delivered to someone with no ability to verify it is not a minor UX issue. These are design choices, and they can be made differently.

We have a rare moment where accessibility can be built into the architecture of how these systems work, rather than layered on after the fact. That window won’t stay open forever. The patterns are hardening. The defaults are being set. And the decisions being made right now will shape who gets to participate in this technology, and who gets left holding a phone, waiting for someone to come help them decode which answer was closest to right.

Let’s not squander it.

• Koenecke, A., Nam, A., Lake, E., Nudell, J., Quartey, M., Mengesha, Z., Toups, C., Rickford, J. R., Jurafsky, D., and Goel, S. (2020). Racial disparities in automated speech recognition. Proceedings of the National Academy of Sciences, 117(14), 7684–7689.
• Leikin, I., Holtz, A., and Hämäläinen, P. (2025). Fairness of Automatic Speech Recognition: Looking Through a Philosophical Lens. arXiv preprint.
• Goldfarb, D., Abreu, R., Zhang, S., Flores, Y., Forero, D., Bradley, A., Stoeckle, M., Feller, E., and Abusamaan, M. (2024). Decoding disparities: evaluating automatic speech recognition system performance in transcribing Black and White patient verbal communication with nurses in home healthcare. JAMIA Open, 7(4).
• University of Illinois Speech Accessibility Project. (2022–present). Speech Accessibility Project, in partnership with Microsoft, Google, Amazon, and nonprofit organizations.
• Google Research. (ongoing). Parrotron: An end-to-end speech conversion model.
• Borg, J., and Lantz, A. (2024). Digital accessibility in the era of artificial intelligence: Bibliometric analysis and systematic review. Frontiers in Artificial Intelligence, 7.
• Euan’s Guide. (2024). AI is transforming the way we live, work, and access information: but is it helping disabled people, or excluding them? 2024 Access Survey.

A reflection from ChatGPT