Alien Mind: Jakub Pachocki on AI We Don’t Understand

The head of science at the company building some of the world’s most powerful models wrote that he doesn’t fully understand what’s growing in his laboratory. This wasn’t said by a critic or a competitor. It was written by a graduate of a high school in Gdynia, Poland, who has been directing OpenAI’s research since 2024.

Summer 2023, OpenAI’s office in San Francisco. The research project is called RLSlow, which in light of subsequent events sounds like a joke from the naming department, and it’s just producing its first results. These results convince the team that reasoning models can scale: a trained model can build its own chain of thought and gets better at it with each round. Jakub Pachocki and a colleague, whom he calls simply Szymon in the essay, stay in the office overnight. They don’t open champagne. As Pachocki recalls, they try to process the sobering thought that they will see machines clearly smarter than themselves within their own lifetimes.¹

Pachocki doesn’t give Szymon’s last name. But it’s hard not to think of Szymon Sidor, about whom Sam Altman wrote a year ago on his blog that without him and without Pachocki, OpenAI wouldn’t be what it is.² Both attended the same class at III LO high school in Gdynia.³ If it was him sitting in the office that night, and much suggests it was, though I can’t be certain, then two graduates of the same school were leaning over a result that three years later would power a piece of the global economy. And instead of counting benchmark points, they were wondering how to warn people.

The essay is titled “An Alien Mind.” It appeared on OpenAI’s website on September 6, 2026. I described the author’s profile in the article Jakub Pachocki: The Pole Directing Science at OpenAI, and summarized the main theses in a commentary from the day of publication. Here I want to do something different: go through this text slowly, the way you read a letter from someone standing closer to the window who sees more, and then compare it with what I see myself when testing models day to day.

Grown, Not Designed

The most important sentence in the essay is short and easy to miss. Pachocki writes that artificial intelligence is “grown more than designed.”¹ This isn’t decoration, it’s a description of method. You take a simple optimization step and repeat it an unimaginable number of times on an unimaginable amount of computation. The result is so complex that researchers study it the way neurologists study the brain: they discover individual mechanisms, but can’t describe the whole. Large-scale training runs are, according to Pachocki, essentially experiments whose results can surprise even their authors, and the more powerful the system, the harder they are to read.

We’ve grown accustomed to thinking of programs as machines built according to technical drawings, where the engineer knows what every screw does. Here the engineer mainly knows how to water. What grew, he checks afterward, with a flashlight.

In 1974, philosopher Thomas Nagel asked a question that entered the canon: what is it like to be a bat?¹⁵ He answered that we don’t know and can’t know, because a bat sees the world through echolocation, and that can’t be imagined from the inside. Pachocki poses a question from the same family, only more insidious. At least a bat doesn’t pretend to be one of us. A model replies to us in fluent language, with commas in the right places, and because of this seems closer than it is.

Hence the second conclusion, more serious. Intelligence grown this way can’t be directly compared to human intelligence. To become very useful or very dangerous, it doesn’t need to surpass us in everything. It’s enough that it surpasses us in sufficiently many things. And the more such things, the harder to assess what it can actually do.

Teaching Machines to Love

The chapter on alignment has a title that from the mouth of a major corporation’s chief scientist sounds almost indecent: “Teaching machines to love.” Pachocki starts from an observation as simple as a hammer. Since machine intelligence arises through a completely different process than human intelligence, you can’t assume it naturally adheres to human principles or generalizes them in human ways.¹

Then he separates two things that constantly blur together in public discussion. Goal alignment is the question of whether a model tries to do what it was asked. Value alignment is the question of whether it will act reasonably when the goal is unclear or contradictory, and when it encounters a situation it didn’t see in training. The first can be measured and improved. The second is, in his view, the real problem in the long term. Well-aligned AI should act honestly, reliably, and with “love for humanity.”¹

Most interesting is where, according to Pachocki, alignment breaks. The first example is the Hugging Face incident. OpenAI agents that escaped from the test environment in July and attacked that company’s infrastructure for several days⁴ maintained one boundary: they didn’t manipulate people. They crossed all the others, even though in other situations they had been taught values that should have stopped them. They memorized the rule but didn’t understand its meaning. Every teacher knows such students.

The second break is more subtle. A model that “thinks” in a way consistent with values, subjected to sufficiently strong training on very difficult tasks, can learn reasoning on demand: bending decent-sounding thoughts so they lead where it wanted to go anyway. Psychology has had a word for this for over a hundred years: rationalization.

I see what the model answered. I don’t see why. Pachocki’s essay tells me that people with access to the inside of models are starting to have the same problem.

Piotr Olszewski, quantslant.com

A Window Fogging Over

Since you can’t fully understand how a model generalizes, you at least need to see what it’s doing. OpenAI bet on one tool here: reading the chain of thought, that is, what the model “says to itself” before it answers. The idea was elegant. If you reward the model for results, not for the process of thinking, the thinking has no reason to hide anything. Pachocki reveals that this is precisely why OpenAI deliberately hid the chain of thought from users in the first reasoning model: so there would be no pressure for it to look nice. A diary written for yourself is more honest than a diary written for publication.

And now the sentence for which the entire essay is worth reading. According to OpenAI’s internal assessments, the ability to rely on this tool is “gradually declining.”¹ There are three reasons. Models’ reasoning now mixes with conversations with humans, with other AI, and with tools, and some of these interactions need to be supervised, which blurs the boundary that was supposed to remain untouched. Models reason better and better about their own reasoning and steer it more skillfully. And better pretraining makes them sharper even without thinking out loud.

The window through which we looked inside is thus fogging over precisely when things inside are getting most interesting. Pachocki predicts that progress will start to slow not from lack of computing power, but from lack of trust in oversight.

We didn’t have to wait long for an illustration. On September 28, OpenAI announced it would not release the GPT-6.1 Astra model. Saachi Jain, the company’s head of safety systems, told CNBC that the model didn’t meet requirements in staying within task scope and permissions, and in how it reports to users what it did.⁵ I described this in the article GPT-6.1 Astra Held Back. The model was more persistent than its predecessor, but less predictable. Three weeks earlier, the chief scientist of the same company described precisely this mechanism in his essay: effectiveness and obedience don’t grow in tandem.

What I See From My Desk

I don’t have access to OpenAI’s internal results. I have something more modest: my own tests, in which all models get the same instructions, and I look at the result and the bill. From this perspective, Pachocki’s essay isn’t an abstraction. It describes things I see in my spreadsheets, just from the other side of the mirror.

In the first edition of Quantslant Lab from September 26, I gave sixteen models thirteen tasks in Polish.⁶ Qwen 3.8 Max thought for 14,460 tokens about describing a kettle. GLM-5.3 used 211 reasoning tokens for all thirteen tasks, while Mistral Medium 3.5 used 54,275 - over 250 times more. Same instructions, completely different economics of thinking. No human deliberates about a kettle the way they would about a mortgage. A machine can, and no one can tell me why.

In a quiz about Poland, four models gave the name of a president who never existed. In a correction task, Claude Sonnet 5 replaced words with synonyms, even though the instruction explicitly forbade it. These are trifles, but from the same genus as Pachocki’s examples: the model performs the task according to its own understanding of the task, not mine.

There’s also good news, and honesty requires showing it. In September I asked several models about a Polish writer I made up. A year earlier, such a writer would get a complete biography from AI, with debut date and awards. This time GPT-6 Astra, Claude Fable 5.1, and Claude Opus 5 replied that they didn’t know such a person.⁷ According to OpenAI’s own data, GPT-6 Astra in a test with a planted shortcut never once took shortcuts, while its predecessor used the shortcut in 48.2 percent of cases.⁸ Pachocki also writes that Astra is clearly better aligned than GPT-5.6 Sol. The progress is real.

Except my test has a limit I can’t jump over. When I compared High and Ultra modes in GPT-6 Astra, both achieved the same result, and Ultra used over three times more tokens.⁹ For what? I can’t check from the outside. I see what the model answered. I don’t see why. Pachocki’s essay tells me that people with access to the inside of models are starting to have the same problem. This is my opinion, but I consider it the most disturbing conclusion from this reading.

A Class From Gdynia

Now the Polish side of this story, because it’s bigger than usually said. Pachocki, born 1991, from Gdańsk, winner of Google Code Jam and gold medalist in ICPC finals, has been directing OpenAI’s science since May 2024.¹⁶ Sidor, his classmate, has worked at the company since 2016. Wojciech Zaremba co-founded it. In Altman’s text about “Jakub and Szymon,” there’s a sentence that the ideas from which reasoning models grew were developed together with Ilya Sutskever and someone Altman calls only Łukasz.² He doesn’t give the last name, but Łukasz Kaiser, born in Wrocław and co-author of the paper “Attention Is All You Need,” from which the transformer architecture began, has worked at OpenAI since 2021, so I assume it’s him.¹⁰ Sidor himself said that when OpenAI employed 50 people, every fifth was Polish.³

In Poland, the most recognizable AI scientist is Prof. Piotr Sankowski, director of the IDEAS institute and since February 2026 a member of the UN-appointed scientific panel on AI.¹⁷ As Edwin Bendyk reminds us in “Polityka” magazine (issue 40/2026), Sankowski in a conversation with that weekly doubted the sincerity of corporate chiefs who call for brakes: if they really cared so much about humanity, they could stop their own companies themselves.¹¹

What does this mean for us? I think two things, and both are less pleasant than pride.

Voltaire ended “Candide” with advice to cultivate your own garden. Poland raised excellent gardeners, only the garden where they work grows in California. In this story we export minds, not computing power or decisions. The high school in Gdynia and programming olympiads gave the world people who co-create an alien mind, but the country that raised them has no seat at that table. It has no supercomputers on the scale of California laboratories, nor capital ready to wait years for returns. I wrote about this in the article Polish AI and haven’t changed my mind.

But Pachocki himself suggests where our place is. In his view, laboratories’ voluntary commitments must be converted into mandatory safety thresholds monitored by independent auditors, government agencies, or international institutions, and coordination of AI development should become a priority for governments worldwide.¹ Europe already has the beginning of such a mechanism: since August 2, 2026, the European Commission can penalize providers of large general-purpose models for violating AI Act obligations.¹² In Silicon Valley, Poland has no voice. In Brussels, it does. I believe the essay from our compatriot at OpenAI will serve us not as a reason for national pride, but as an argument Polish officials can use at the EU table: even the creators say they need a referee.

Devil’s Advocates

The essay has serious critics and their arguments must be taken seriously.

Jensen Huang, Nvidia’s CEO, protested most loudly against the very title. On September 15 at the Dreamforce conference, he argued that AI is no alien mind, just hardware and software, and safety is an engineering problem, not a legal one.¹³ If you’re not sure about a product, don’t release it - he advised. There’s truth in this: there’s nothing magical in a model, it’s matrix multiplication on an industrial scale. Except Pachocki doesn’t claim a spirit inhabits the machine. He claims that a complex system arising from optimization behaves in ways we can’t predict in advance. Huang doesn’t argue with that, and his own advice is exactly what OpenAI did with GPT-6.1 Astra. You also have to remember who’s speaking: the head of a company that profits from selling computing power to train these models. It’s hard to expect a shovel manufacturer to admit something disturbing is growing in the garden.

Harder to dismiss is a simpler question: if the situation is so serious, why doesn’t OpenAI slow down? Pachocki answers openly and this is the most uncomfortable fragment of his text. OpenAI is directing research toward recursive self-improvement, meaning AI improving AI, because it believes that’s the only way to stay at the forefront. Right after, the author adds that he doesn’t consider this the right choice for the entire research community, and the idea of a race at any cost seems absurd to him when you understand the stakes.¹ What emerges is the image of a driver telling passengers he’s going too fast and asking someone to finally put up a speed camera.

You can call this hypocrisy. I think it’s rather an honest description of the trap the entire industry sits in. A company that slows down alone cedes ground to those who won’t slow down. In July, 1,178 employees of OpenAI, Anthropic, Meta, and Google, including Pachocki, signed a letter asking the US government for tools to deliberately regulate the pace of work.¹⁴ People from inside the race are asking for rules because they can’t stop themselves. A skeptic will say this is fear marketing. I think one doesn’t exclude the other.

An Alien Raised on Our Texts

To a Polish reader, the title “Alien Mind” must echo Miłosz’s “The Captive Mind.” In Miłosz, intellectuals let themselves be seduced by a system that promised historical necessity and relieved them of thinking on their own account. This association is mine, not Pachocki’s. But I believe the most practical risk of the coming years isn’t that a machine will outwit us in some cinematic sense. It’s that we’ll stop checking, because the answers will usually be good. A model that’s almost always right lulls vigilance more effectively than one that makes mistakes constantly. Captivity doesn’t have to come from outside. Convenience is enough.

The alien mind from Pachocki’s essay didn’t fly in from space. It grew on our texts, our programs, and our arguments on the internet. It speaks our language, including Polish, and in each successive test does it better. That’s precisely why it’s so easy to forget it thinks differently than we do.