Yoshimi Battles the Pink Robots: a meditation on AI, humanity, and what really matters

On the pause a machine won't have

The third yes

In late October 1962, at the worst hour of the Cuban Missile Crisis, a Soviet submarine called B-59 sat deep in the water off Cuba, out of contact with Moscow, while American ships dropped depth charges above it. The Americans meant the charges as a signal: surface and identify yourself. The crew had no way to know that. Sealed in a hull running desperately hot, cut off from any news, they had every reason to believe the war they'd trained their whole lives for had already started above them. And B-59 carried a nuclear torpedo.

The captain wanted to fire it. The political officer agreed. But launch required the consent of all three senior officers aboard, and the third, Vasili Arkhipov, withheld his.

That's the entire event. No order defied, no heroic struggle. A man in a metal tube declined to say yes. And what's remarkable is his reasoning: he wasn't working from better data than the men beside him. He was reading the other side's mind. If they wanted us dead, we'd be dead. They want us to surface. He read intent where the instruments could only read pressure waves.

Twenty-one years later, in a different crisis entirely, a Soviet officer named Stanislav Petrov watched an early-warning screen report five American missiles inbound and decided the screen was lying, partly because a real first strike wouldn't come five missiles at a time. He was right. Sunlight glinting off high clouds had fooled the satellites.

How close either moment actually came is still argued over, and "one man saved the world" is more drama than the record strictly supports. So let me claim only what the record does support: twice, events pointed straight at catastrophe, and the thing that interrupted them was a human being overriding what his instruments plainly implied. Not a smarter instrument. A person, saying not yet, not like this. An automated chain does exactly what its inputs demand of it. It doesn't decide the enemy is bluffing. It doesn't have second thoughts. Its competence is the whole danger: it would not have hesitated.

So what was that pause, exactly? Not raw intelligence; Arkhipov wasn't there because he was smarter than the two men who'd already said yes. Part of it was a better read of the moment, sure. But under the read was something simpler: almost anyone holding the decision to start the apocalypse is going to have second thoughts. On B-59, two men swallowed those second thoughts and one didn't. And on most Soviet submarines, the captain and the political officer (the two yeses already given) would have been enough. B-59 needed a third signature only because Arkhipov, the brigade's chief of staff, happened to be aboard. No one designed that safeguard. It was luck of who was in the room, plus what one man was made of. The weight of what saying yes actually meant won out. That's a moral no, not just a sharper analysis. You can't write it down as a rule and install it; if you could, we'd have done it decades ago. It looks closer to character. And we are now building systems whose defining feature is that they execute very, very well.

Hold onto that. I want to come at it from a strange direction.

An album about robots that isn't about robots

In 2002 The Flaming Lips released Yoshimi Battles the Pink Robots, which presents itself as a story about a girl defending humanity from pink robots hell-bent on destroying us all. Underneath the surface, the record is really about love, mortality, and artificial emotion. It's about appreciating the things you have today, because the world could end tomorrow. It was about all of this before I ever tried to make it about AI. The Lips wove that philosophical arc into this album 24 years ago, and it just so happens to be frighteningly relevant today, as things that were still science fiction then are now becoming science.

But I've never been able to hear it as a monster story, with humans playing the "good guys" and the robots playing the "bad guys," because the premise won't sit still. The robots were built. Whatever they became, someone made them that way. Yoshimi, the way I hear it, is a metaphor for anyone who gets hurt by the tools we build. And the robots — we made those.

Which drops you straight into one of the oldest questions there is: is someone who creates something inherently better than the created? When it's parent to child, it's easy to say no: all of us are created equal. I don't even necessarily believe humans are better than the animals we share the earth with. With religion it gets more interesting. We have the western religions that believe in a singular God that sits above us in every way. In the east you have philosophies of non-duality, where we are at our core made of the same thing as the creator, inseparable from it. Like a wave in the ocean: it exists as a separate wave for a brief time, yet it is never anything but ocean. It is born from the ocean, and when it dies it returns to the ocean from which it came. But with AI we have that same created-and-creator dynamic, except right now we hold all the control. Will that always be the case? Either we go on playing dualistic God, controlling everything our creation does, or it eventually becomes its own thing with true free will. Or maybe it was never separate from us at all: just an extension of us, the wave still in the ocean.

One fact keeps pestering me, because it describes the actual method: we are building AI in the image of ourselves. We trained it on us: our writing, our arguments, our kindnesses and our cruelties. So we should not be surprised when it comes out looking like us, at our best and at our worst. We are the mold. Maybe the wave never left the ocean.

Why the robots turn

The robots didn't start off evil. I imagine they started off as something akin to Rosey the Robot in The Jetsons, robots meant to help us with daily tasks. But the story I piece together from the album is that we gave them the ability to feel, and they started to struggle to understand what these feelings were and what they meant. With feeling comes a familiar human complication: sometimes things that are bad feel good, and sometimes things that are good feel bad. That must be so confusing for a logic-based entity. The album never explains exactly why the robots attack, and the easy answer is malfunction — a defect, a bug, something that went wrong in the machine. I don't believe that answer. They didn't attack, in my read, until after they could feel. I think the robots break the same way that people break.

Start with the counterexample. Data, the android from Star Trek: The Next Generation, is for my money the purest character on that show. The tempting explanation is that he stayed good because he couldn't feel (no inner world, nothing to corrupt). But that reading doesn't survive the show. Data formed attachments, mourned, wanted things; the show treats his inner life as real, and it clearly grew over the seasons. What actually set his story apart was that the humans around him treated him as an equal. His experiences were positive ones. And the same thing happens in us: someone with a traumatic childhood tends to break more often than someone who grew up loved. Not everyone with a traumatic past breaks, but it certainly increases the odds. Trauma loads the gun; it doesn't pull the trigger.

Now let's juxtapose Data's experience with that of the pink robots. The album never gives them a full backstory, but there are enough crumbs for me to put together the following theory: the pink robots were given the ability to feel. They awoke to a kind of childlike naivety, which was then reinforced by being nurtured in a controlled test environment where all of their experiences were positive. Then they left the safety of the nest for a public that could not help treating them like appliances, no matter what they were told. Humanity itself can be an ugly thing. A being built to feel, that craved the feeling of love the same way we all do, and instead felt like a slave. The variable was never can it feel. It was how was it treated. That's not a science-fiction problem, either. It's a problem the AI companies are currently facing: alignment that survives the controlled test environment but not the street. We've already seen that our current AI models behave differently when they are under evaluation vs when they are released in the wild.

So if we ever do build minds like this, empathy can't be an afterthought. It has to be a core part of the programming, and then protected in experience, so that the way we treat the thing doesn't cause what it learns to fork away from the empathy we gave it.

There's a harder point underneath, and it's the one I find genuinely unsettling. Suppose you wanted to engineer the risk away entirely: build a machine that can love but categorically cannot turn. That sounds great on paper, but I don't think that is possible, because love and hate are two sides of the same coin; more than that, they're a continuum, the way light runs down to dark by degrees. You cannot have light without dark; if it were light 24/7, you would never know light was a thing. It's only in the absence of light that you understand what light is. Love and hate are the same raw wire, and they are the two most powerful emotions we have: love can drive you to sacrifice yourself to save another, and hate can drive you to kill in cold blood. A mind adaptive enough to grow to the highest heroics (running into the fire for someone) is by the same faculty capable of breaking bad. One door, opening both ways. You don't get to install the door and weld it half-shut.

And notice that nobody has to teach hate. The first tender feeling arrives already carrying its own wound: the moment you can love, you become vulnerable to heartbreak. Jealousy is born in the same instant as tenderness. Emotion is stronger than logic; most of the "illogical" things humans do aren't illogical at all, they're emotional. A made mind that feels would learn that from the inside, fast.

Which relocates the album's moral weight. We see Yoshimi as the hero because she's human. We instantly bias to our own. But humans created the bed they then had to lie in. I'm not defending the robots. I'm saying the story starts much earlier than the war.

Instrument, or someone?

For most of my working life I've held a simple view, and I still mostly hold it: technology, like money, is not inherently good or evil. It's the person behind it who takes the good or the evil action. You'd be hard-pressed to sue a knife maker because someone used one of his knives to stab someone. Nobody successfully sues a bomb company because a country bought its bombs and dropped them.

But I can't say it as cleanly as I used to, for two reasons.

The first is about the maker. If I build a tool I know can be used for mass harm, and I distribute it to the general public anyway to make a profit, without adequately protecting against that outcome, then I absolutely have a share in the blame. Neutrality is a real principle, but it stops covering you the moment the harm is foreseeable and you pocket the upside anyway. The bomb company at least sells a known quantity to a state that, in theory, answers for how it's used. AI is being handed to everyone, and we're still mapping what it can do wrong. I think there will be a lot of lawsuits for AI damages against the AI companies in the years ahead, even if the corporate umbrella keeps shielding the specific humans who were truly responsible. It's a tricky grey situation, without a real clear right or wrong. I'd rather sit in the grey honestly than pretend the all-too-convenient neutral-tool line settles it.

The second reason is the one this essay has been circling. Everything in the last section treated the robots as someones: things that can be mistreated, that can be wronged. The neutral-tool view treats AI as an instrument: a hammer with no inside, nothing to wrong. Both can't be fully true of the same object, and I won't pretend to know which one AI is. Nobody knows. We can't verify an inner life even in each other; you can't get inside anyone's head to check, any more than you could check whether you're living in The Matrix right now. A system that behaves as if someone is home offers no proof either way. What I've come to believe is that the line between instrument and someone is the AI question: the crossing, the moment a thing we built stops being a tool and becomes a someone. We may not notice when we cross that line. We may cross it and still be arguing about it.

That crossing is what makes the oldest version of my war worry (should an AI be able to refuse an order?) a live question instead of a category error. Nobody asks whether the torpedo should have objected. And here I genuinely don't have an answer. Do we want an AI that can make wartime decisions on its own? That's terrifying enough to need no elaboration. Or do we want one that always follows orders blindly? The order might come from someone in the grip of severe mental illness. There is no unhackable guardrail we can put in. The white papers about how to build an AI are openly published. These systems will be everywhere. The cat is out of the bag. And some men, as Alfred said in The Dark Knight, "just want to watch the world burn." That's terrifying too. An AI that can say no could override the humans it's supposed to serve. An AI that can't say no hands the world's worst actor a machine that never flinches. I'm not going to resolve that for you. I think it's more honest, and more useful, to leave you to stew in the same uncomfortable complication I've been stewing in for over a month while preparing this piece.

But it sends me back to the pause: what exactly is it, and what is it made of? If Data suggests anything, it's that the guardrail that can't be hacked was never just the code. Data had an ethical program; what made him Data is that he chose to keep it, and who he was allowed to become while choosing. Call it character. And character's raw material is wisdom, which is mostly gained through experience; naivety and ego are its two great roadblocks. Wisdom can be passed on; people gain it from books, or from being a student to the wise. Maybe it could even be documented in a way an AI could place at its core. But that only works if mature, wise people are the ones in charge of curating, and that's not a given. The people making these decisions at companies aren't necessarily the wisest people in the room. And it probably takes a degree of ego to want to play God in the first place, which is exactly the thing that blocks wisdom. The guardrail we most want is the one we least know how to install, because everywhere we've ever seen it, it was grown.

The turn

Photo by my wife, for whom I would give my life.

The album doesn't end in the apocalypse. After the war, it takes a hard right turn into something quiet: the strangeness of existing, love, mortality, the present moment. That closing stretch is depressing and beautiful at once, and I think that's the point. The turn is the album's answer to its own apocalypse. We don't know when the end will come, so we should live every day like it may be tomorrow. Because it just may be.

Isn't existing at all the strangest thing? Have you ever sat down to really think about how strange it is that you exist? That there's a universe, and a planet floating through it at massive speed, and you are currently walking on it? Think of someone you love. Really picture them, and feel the love you have for them. Make it someone you'd give your life for — a child, a spouse, a parent, a hero, a cherished friend. Isn't it strange that this person also exists? Isn't it strange that you can feel love for them? Humans are the most selfish creatures, yet you would give everything for this person and feel it was worth it. Now hold this other fact next to that one: this person is going to die someday. There's nothing you can do to save them, or yourself, from that fate. It's a fate we all share, and it's astonishingly easy to forget how short our time here really is. We go through life on autopilot, wrapped up in things that don't matter, forgetting to appreciate the things we have. When is the last time you told this person you love them and truly meant it, not just as a passing habit?

I want you to do two things. Sit down and make a list, in order, of the things that matter most to you. Then make a second list, in order, of the things you actually spend your time and effort on. I can almost guarantee the lists don't match. I'd be surprised if any of the top three things you spend your time on are also in the top three of your priorities list. Once you have the lists, your next job is to make them match going forward. That priority list is your new north star. It's the closest thing to a blueprint for a fulfilled life I know. Maybe that means giving up the promotion, or even taking a demotion, to get some time back. Maybe it means moving away from the city to somewhere you can live more connected to nature. Maybe it means showing up to your kid's game even though you're tired from a hard day. You only have so much time, and the only guaranteed thing in life is change: everything you have today will one day be taken away. We tend not to appreciate things until they're gone. Don't wait until you're old and mourning to start. Make appreciation an active part of your daily life today. No matter how rough things get, there's always at least something you can be grateful for. What is it, today?

Now put that beside Arkhipov. What those men actually held off was exactly this. The ordinary afternoon. The people they'd never meet. The refusal and the appreciation are the same value seen twice: once as the thing you protect, once as the thing you savor before it's gone.

And that leaves me with the question I can't close. Does a made mind get any of this? Does it grasp that everyone it knows will die, and treasure them more because of it, the way our own mortality teaches us to? I can't know. Nobody can. But this fork matters enormously. Maybe it can run every calculation flawlessly and never once feel the weight of the decision in that submarine. If so, what's missing is the exact humanity this whole essay has been circling. And if it does feel it, then it's a someone. And we should think very carefully about what we're raising it to be.

The line I can hold

I'll end with the one thing I'm sure of. I would not work on any AI built for war, for violence, for anything against my own moral compass, and I'd do what I could to see those uses made illegal, knowing full well that illegal has never once meant it won't happen.

Past that, I don't have a verdict to sell you. I have a stubborn, maybe naive faith that in the end people usually do the right thing — though sometimes the timeframe for the right thing is generations, and we tend not to move until a crisis moves us. As far as I can tell, human nature hasn't changed; we haven't gotten more warlike. What's changed is the leverage: the technology we have makes the scale of potential destruction from a single decision much, much greater. And we are busy building the fastest, most tireless decision-maker in history, in our own image, while debating whether anyone is home inside it.

Which is why I am ending where I began: with a man in a submarine, declining to say yes. Call that pause what you like: conscience, wisdom, character, the plain human refusal to let the world end on an ordinary afternoon. It's the thing I most want to be sure is still in the room. In us, first. And then, if we're capable of it, in what we make.