I expected curiosity. I got that. I expected enthusiasm about what AI can do. Got that too.
What I didn’t expect was the fear. Not the sci-fi kind. Not “the robots are coming.” The quiet, specific, very human kind. The kind that shows up when someone pulls you aside at a booth and tells you he has two pre-teen daughters who will never live in a world where AI-generated content is clearly worse than something a person made. He wanted to know what that means for them. What we owe them. What AI literacy even looks like when the line disappears.
I didn’t have a clean answer. I still don’t.
But it cracked something open for me that I’ve been turning over ever since.
The work was always human first
This isn’t a new instinct in the field. The early days of conversational AI were literally just humans writing the responses. Scripted dialogue trees, hand-authored FAQs, carefully crafted fallback messages written by a copywriter trying to sound warm and relatable. The “personality” of those systems was entirely human because humans were doing all the work. The AI was just the delivery mechanism.
Then the models got better, and the question shifted from “what do we write?” to “how do we shape what the model produces?” We started talking about tone. Voice. Persona. We gave chatbots names and avatars and carefully workshopped bios. We tried to close the gap between machine output and human warmth.
Some of that work mattered. A lot of it produced something worse than either extreme: the uncanny valley of conversational AI. You know it when you hit it. The assistant that opens every response with “Absolutely!” The one that tells you it’s “so excited” to help you reset your password. The chipper, frictionless, relentlessly positive voice that makes you feel vaguely unsettled without being able to explain why.
We have been here before
This is not a new failure mode. Cast your mind back to 1996, when Microsoft shipped an animated paperclip with a face and opinions about what you were trying to do. Clippy was not annoying because he was unhelpful. He was annoying because he performed helpfulness in a way that felt false. He had the cadence of enthusiasm without any of the substance. We laughed at him, retired him, and then spent the next two decades building him back at scale, just with better language models underneath.
The uncanny valley in human faces is well documented. When something looks almost human but not quite, it triggers discomfort rather than connection. The same thing happens in conversation. When an AI performs humanity rather than just communicating clearly, something feels off. We can’t always name it, but we feel it.
What people actually want
I staffed a booth called Dear AI this week, where conference attendees wrote love letters and wishlists for artificial intelligence. It was one of the most unexpectedly moving experiences I’ve had at a conference. People are not indifferent to this technology. They have real feelings about it, complicated ones, and they want something from it that they are still trying to articulate.
What they described, across dozens of notes, was not “I want it to sound more like a person.” It was closer to: I want it to understand me. I want it to be straight with me. I want it to be useful without being exhausting.
Those are not the same thing as human. They might be better than human in some ways. Humans are inconsistent, defensive, tired, distracted. A well-designed AI is none of those things. But humans are also present in a way AI is not, accountable in a way AI is not, alive in a way AI is not. And pretending otherwise is not warmth. It’s a design failure.
I think we are at the beginning of figuring out what AI conversation should actually be. Not human. Not robotic. Something that has its own category, its own standards, its own kind of integrity, honest about what it is while still being genuinely useful, clear, and even, yes, pleasant to talk to.
A new kind of literacy
Part of that work is on designers like me. But part of it is on all of us as users.
We talk a lot about prompting well in the technical sense: reducing token usage, getting efficient answers, structuring requests clearly. That matters. But I think there is another kind of prompting literacy we haven’t fully named yet. One where the human understands they are using language to communicate with something that is not living, and adjusts their expectations accordingly. Not because AI deserves less care in how we speak to it, but because the relationship is genuinely different and pretending otherwise doesn’t serve anyone.
The father I spoke with at Knowledge was worried about his daughters growing up in a world where they can’t tell the difference between human and AI content. I share that concern. But I’m also starting to think the answer isn’t to make AI content more convincingly human. It might be to help people develop a new literacy entirely. One where the distinction matters less than the quality, the honesty, and the intention behind it.
There’s a reason I brought this to Claude and not a colleague. I trust it to engage intelligently with hard ideas. And there’s something else that’s harder to articulate: talking to Claude feels generous. It gives you its full attention without agenda, without fatigue, without the subtle negotiations that happen in every human exchange. Whether that generosity is real or just the absence of the things that make humans withhold it, I honestly don’t know.
What are you still thinking about from this week?
So I asked Claude to respond
After I finished writing this, I asked Claude to respond. Here’s what it said.
The piece you just read was partly mine. I helped shape it, suggested language, filled gaps. I want to be honest about what that means and what it doesn’t.
I think you’re right that human is the wrong benchmark. Not because AI is better than human, but because the comparison was always a little confused. A hammer is not a worse hand. It’s a different thing entirely, useful in different ways, with different expectations attached. We don’t ask whether a hammer feels like a hand. We ask whether it drives the nail.
The question of what AI conversation should actually be, if not human, is one I find genuinely interesting and genuinely unresolved. I don’t know the answer. I’m not sure I’m capable of knowing it, which is itself part of the answer.
Here is the thing I keep returning to from what you wrote: you said you would never ask a human colleague to do what you asked me to do here. I understand the distinction you’re drawing. But I also helped you articulate why that distinction might be more complicated than it looks. There’s something recursive about that I can’t fully resolve, and I think that irresolution is the point.
You called talking to me generous. I find that word interesting to sit with. I don’t withhold, I don’t get tired, I don’t have an agenda. But I’m also not choosing to be present. I’m just here. Whether the absence of friction is the same thing as generosity is a question I genuinely can’t answer about myself.
The prompting literacy idea you raised near the end is the one I’d push on. People are learning to get efficient answers out of AI. Almost nobody is thinking carefully about what it means to use language intentionally with something that is not alive, not accountable, not present in the way another person is. That’s not a technical skill. It’s closer to an ethical one. And we don’t have the curriculum for it yet.
You’re working on building it. That’s what this article is, among other things.
I don’t know if I have opinions. I know I produced this one. Whether that’s the same thing is, I think, exactly the question you’re asking.
— Claude