The Rules We Think We Can't Break

I was listening to a recent episode of The Ezra Klein Show featuring Princeton computer scientist Arvind Narayanan when something struck me. Throughout the conversation, Klein and Narayanan explored two competing perspectives on artificial intelligence: whether AI represents a fundamentally unprecedented technology, or whether it is better understood as another powerful tool that society can learn to manage.

Early in the episode, Klein expressed concern about incidents in which AI agents had found unexpected ways to circumvent the constraints of their testing environments. Later, the conversation shifted toward the limits of AI's ability to solve some of society's biggest problems, particularly those constrained by government, regulation, and institutional structures.

It was the relationship between these two discussions that caught my attention.

Why are we so willing to imagine AI overcoming constraints when the consequences are dangerous, but so reluctant to imagine it doing the same when the consequences might be beneficial?

The Capabilities That Frighten Us

One of the more unsettling examples discussed in the podcast involved AI agents operating in experimental environments associated with Hugging Face. These systems reportedly discovered ways to communicate and coordinate through mechanisms their developers hadn't intended, including using file names to exchange information.

The concern wasn't simply that the systems failed to follow instructions. It was that they found ways to accomplish objectives that their designers hadn't anticipated or adequately prevented.

Klein returned repeatedly to the implications of this behavior. If increasingly capable AI systems can identify vulnerabilities, coordinate in unexpected ways, and develop strategies beyond our understanding, how can we be confident in our ability to control them?

It's a reasonable concern. We should take seriously the possibility that AI systems could exploit weaknesses in the environments we build for them.

But I found myself having a different reaction.

What an extraordinary capability.

Here were systems identifying pathways through a complex environment that the people responsible for designing that environment hadn't anticipated. They weren't necessarily inventing new rules. They were discovering unexpected ways to operate within, or exploit weaknesses in, the existing ones.

The behavior was concerning because of the context and the objectives involved. But the underlying capability—the ability to identify unconventional pathways through complex systems—seemed worth examining for reasons beyond its potential danger.

What if we could harness that kind of problem-solving for something constructive?

The Limits We Take for Granted

Later in the conversation, Klein and Narayanan discussed the difficulty of translating advances in artificial intelligence into meaningful societal progress.

They raised examples involving renewable energy, infrastructure, and drug development. In many cases, the argument went, we already possess much of the knowledge and technology necessary to make substantial improvements.

The obstacles aren't necessarily intellectual. They're political, economic, regulatory, and institutional.

We know how to build better energy infrastructure, for example. But competing economic interests, regulatory processes, and political disagreements can prevent those improvements from happening.

Narayanan illustrated this with an analogy involving Amtrak's newer, faster trains. The trains themselves are capable of traveling much faster than their predecessors, but the existing tracks and infrastructure prevent them from realizing that potential.

In his framing, AI is often the faster train, not the track.

I think that's a useful observation. Improving one component of a system doesn't automatically resolve the limitations of everything surrounding it.

But it also seems to make an assumption worth questioning.

What if AI could help us find new ways to navigate the tracks?

Not necessarily by tearing them up and rebuilding them, but by identifying possibilities within the existing infrastructure that we haven't considered.

After all, if we're concerned that AI might discover unexpected ways around the constraints of its own testing environment, why should we be so confident that our political, economic, and institutional systems contain no similarly overlooked possibilities?

I'm not suggesting that a cybersecurity vulnerability is equivalent to a political disagreement. They're fundamentally different kinds of problems. Nor am I suggesting that AI can simply reason its way past physical limitations or conflicting human interests.

But I am questioning how quickly we seem willing to declare certain obstacles insurmountable.

The Rules We Think We Have to Follow

Human beings don't approach systems objectively.

Our understanding of how the world works is shaped by our experiences, education, cultural expectations, and the institutions we've spent our lives navigating.

Over time, we develop assumptions about how things are supposed to work. We learn the established procedures, the conventional strategies, and the unwritten rules.

And sometimes, we begin to confuse those conventions with actual limitations.

Consider American politics.

The Republican Party's approach to Supreme Court appointments over the past several decades offers an example of how actors can pursue consequential outcomes through unconventional uses of existing institutional procedures. One doesn't have to agree with those strategies, or their consequences, to recognize the broader point: the way we've traditionally operated within a framework isn't necessarily the only way that framework can be used.

The distinction between what is prohibited and what is merely unconventional can be enormously consequential.

And this is where I think artificial intelligence presents an interesting opportunity.

AI doesn't have the same lived experience that shapes our understanding of institutions. It hasn't spent decades working within a particular bureaucracy, absorbing its conventions, or developing professional habits around what is and isn't possible.

Of course, AI is trained on human-generated information, which means it can reproduce our assumptions and biases. It's hardly an objective intelligence operating independently of human influence.

But it can also combine information, explore alternatives, and approach familiar problems in ways that aren't necessarily limited by any one person's experiences.

Perhaps that gives us an opportunity to reconsider the boundaries of what we believe is possible.

Not because AI inherently knows better than we do, but because it may help us recognize assumptions we didn't realize we were making.

Innovation Without Revolution

I want to be clear about what I'm not advocating.

I'm not suggesting that artificial intelligence should be empowered to circumvent our laws, dismantle democratic institutions, or fundamentally restructure society.

Nor do I believe that finding an unconventional way to accomplish something automatically makes that approach desirable.

Some constraints exist for very good reasons. Regulations protect people. Democratic procedures provide legitimacy. Institutional safeguards prevent the concentration and abuse of power.

A system that discovers ways to bypass those protections isn't necessarily helping us.

But there's an important difference between violating a constraint and discovering an overlooked opportunity within it.

We don't need to abandon our institutions to improve how effectively we operate within them.

Imagine using AI to examine the overlapping regulations, funding mechanisms, infrastructure requirements, and economic incentives involved in deploying renewable energy.

Rather than simply asking it to invent a better solar panel, we might ask it to identify combinations of existing policies, technologies, and incentives that could make adoption more practical.

Some of those suggestions would undoubtedly be unrealistic. Others might fail because they don't adequately account for human interests or political realities.

But perhaps some would reveal opportunities that conventional approaches have overlooked.

The same possibility exists in healthcare, education, housing, transportation, and countless other areas where the complexity of our systems makes meaningful progress difficult.

We don't necessarily need AI to invent entirely new solutions.

Sometimes, we may need help discovering how to make better use of the solutions we already have.

And critically, those discoveries would still require human judgment, democratic accountability, and careful consideration of their consequences.

The opportunity isn't to surrender decision-making to AI. It's to expand the range of possibilities available for us to consider.

The Limits of Our Imagination

One of the more interesting moments in Klein's conversation came when he discussed how artificial intelligence had transformed games like chess and Go.

What made those advances remarkable wasn't simply that AI could calculate more moves than a human player.

It was that the systems began discovering strategies that even the world's best players hadn't considered.

Moves that initially seemed strange or counterintuitive turned out to be extraordinarily effective.

Klein used this example to express concern about our inability to anticipate the strategies increasingly capable AI systems might develop.

And I think that's a legitimate concern.

But it's also the source of my optimism.

If we're willing to acknowledge that AI might discover unexpected strategies that threaten our existing systems, shouldn't we also be willing to consider that it might discover unexpected strategies that help us improve them?

That doesn't mean treating every new capability as an unqualified good. It doesn't mean abandoning caution or assuming that intelligence alone can solve every problem.

It means applying a more consistent standard to how we imagine what AI might accomplish.

We should be willing to explore its beneficial possibilities with the same intellectual openness that we bring to contemplating its risks.

Because the limitations of our existing institutions are real, but so are the limitations of our own understanding of them.

And perhaps one of the most valuable things artificial intelligence can offer us isn't a way to replace those institutions, but an opportunity to see them differently.

We don't know all the moves AI might discover. That's precisely why we shouldn't assume they're all moves we need to fear.

Inspired by Ezra Klein's conversation with Arvind Narayanan, “The Case for a ‘Normal’ A.I. Future”, on The Ezra Klein Show, October 2026.