I recently read Eliezer Yudkowsky’s Inadequate Equilibria: Where and how civilizations get stuck. I managed to take away some interesting ideas, but I was left with the overwhelming impression that he wasn’t talking to me, that he has only really been challenged by other economists. This is not an indictment of the book exactly; not everything needs to be written for me. But I do think his analysis is impoverished from an overdose of market ideology.
The question Yudkowsky sets out to answer is “When can we expect to do better than our society (at some given problem X)?” For instance, beating the stock market, making a successful startup, or coming up with an undocumented treatment for a medical condition. It is tempting to answer “almost never” since society is big and composed of many individuals who are much more capable at any particular task than you or I. To this Yudkowsky responds that systems may be broken or stuck in such a way that all those brilliant people cannot actually effect X.
They might not be in a position to take action X (or benefit from action X), such as when many economists know that the central bank of Japan has a bad policy, but are not themselves in charge. Or they might be constrained, such as academics who have to get published and cited in order to succeed and be taken seriously.
One of his central points is that these constraints can enter into the picture even if everyone is working towards the same goal. For instance, if everyone agrees that Facebook is terrible, and would prefer a different social media site, they may still not switch over. Because any one person who switches over has no access to their friends there, which is the whole purpose of social media. There is no fundamental constraint on people’s behaviour; sometimes it arises from the structure of the system, and sometimes from the state of the system.
He then proceeds to explain different ways that the system can get stuck, and how to identify systems that are likely stuck. This brings us back to the original question: “When can we expect to do better than our society?”
I think the ideas that I’ve outlined above are true and worth-while, and some of the broken systems Yudkowsky identifies are new to me. I really do feel like he taught me something. So why do I feel like an eavesdropper; who is Yudkowsky talking to?
Well, part of that question he answers himself in the last chapter:
I expect this book to be of much more use to the underconfident than the overconfident
But not just any underconfident person. In particular, someone who thinks that societal systems (and in particular markets) will always produce optimal outcomes. And as individuals we can’t improve on them significantly. His task in this book is to argue that even big systems, even markets, can be inadequate.
Most of chapter 3 consists of a dialogue between 3 main characters:
- The Visitor from a Better World;
- Simplicio, who is attending a major university but hasn’t taken undergraduate economics;
- Cecie, the Conventional Cynical Economist
Cecie and the Visitor together describe Yudkowski’s point of view: how our world is stuck, and what it might look like if it weren’t. It took me a while to realize that Simplicio doesn’t represent the reader. Simplicio represents an individualist point of view, summed up perfectly in this line:
SIMPLICIO: I feel like you’re placing too much blame on system-level issues, Cecie, when the simpler hypothesis is just that the people in the system are terrible: bad at thinking, bad at caring, bad at coordinating.
Simplicio agrees that society is broken, and that we as individuals can do better; that’s not the audience Yudkowsky is addressing. I think the imagined reader is instead someone who has to deal with a lot of Simplicios in their life. Someone who believes in the invisible hand of the market, and argues with people who think individual incompetence and malevolence leads to a broken system.
Yudkowsky’s purpose in this dialogue is to show that one can disagree with Simplicio, and still think the system is broken. But no one in this dialogue, Yudkowsky (in the form of the Visitor + Cecie + himself later on), Simplicio, or the imagined reader advocates for anything other than markets.
When Yudkowsky begins to explain efficient markets, he makes a very important observation.
In a pre-market economy, when you offer somebody fifty carrots for a roasted antelope leg, your offer says something about how impressed you are with their work hunting down the antelope and how much reward you think that deserves from you. If they’ve dealt generously with you in the past, perhaps you ought to offer them more. This is the only instinctive notion people start with for what a price could mean: a personal interaction between Alice and Bob reflecting past friendships and a balance of social judgments.
In contrast, the economic notion of a market price is that for every loaf of bread bought, there is a loaf of bread sold; and therefore actual demand and actual supply are always equal. The market price is the input that makes the decreasing curve for demand as a function of price meet the increasing curve for supply as a function of price. This price is an “is” statement rather than an “ought” statement, an observation and not a wish.
In particular, an efficient market, from an economist’s perspective, is just one whose average price movement can’t be predicted by you.
Efficient markets, in this view, are not necessarily a better or even a good way of organizing things, they are just a way of organizing things, that we find in society, and we can say something about what properties they do or don’t have. But this idea gets lost pretty fast, and about halfway through the book, buried in a parenthetical, the market ideology really shines through.
(Note the inevitable comparison to financial markets—the one part of civilization that worked well enough to prompt an economist, Eugene Fama, to come up with the modern notion of efficiency)
Not only are efficient markets good, they are the only part of civilization that works well!
In reality, the opposite is often true. Even constraining ourselves to the narrow world of capitalist markets, sometimes protectionism and shielding of infant industries is the right policy. But that has the express purpose of making an inefficient market!
What’s so fascinating to me is that if you want to read it that way, this book actually presents a fairly good criticism of markets. It explains some of the ways markets go wrong and fail to provide us with good outcomes, but nowhere does Yudkowsky seem to realize that the solution might be something other than markets.
Yudkowsky seems to have a limited imagination when it comes to what systems are even possible. He imagines pretty much only equilibrium systems composed of rational, self-interested agents competing for some scarce resource in an unstructured arena.
Let’s take those one at a time.
To be more precise, he describes systems that are essentially or usually at equilibrium, potentially interrupted by moments of non-equilibrium movement. But in fact, non-equilibrium processes abound in the world, from cyclic processes such as pork cycles, speculative bubbles, and fads, to monotonic ones like technological advancement, and population growth.
Let’s be clear about just how ubiquetous he thinks equilibria are. I’ll let him speak for himself, from the dialogue in chapter 3.
CECIE: …the frustrating parts of civilization are the times when you’re stuck in a Nash equilibrium that’s Pareto-inferior to other Nash equilibria. I mean, it’s not surprising that humans have trouble getting to non-Nash optima like “both sides cooperate in the Prisoner’s Dilemma without any other means of enforcement or verification.” What makes an equilibrium inadequate, a fruit that seems to hang tantalizingly low and yet somehow our civilization isn’t plucking, is when there’s a better stable state and we haven’t reached it.
VISITOR: Indeed. Moving from bad equilibria to better equilibria is the whole point of having a civilization in the first place.
The whole point of having a civilization.
In the dialogue in chapter 3, Simplicio is all about blaming individual incompetence, irrationality, and malice. Cecie’s response seems to be that we don’t need to appeal to individual irrationality to explain the behaviour of the system. But why is rational self-interest the right null hypothesis?
In one of the quotes I cited above, Yudkowsky imagines a barter system that does not behave according to this principle. People work to maintain complex social relations that don’t have to do with optimizing anything. Does he imagine that that’s true only of “pre-market” societies, and now we’re all enlightened? I’m sure he’s aware of how irrational most people are. In fact, Yudkowsky knows that people don’t have utility functions; he says as much in a talk from 2016. Hell, this book itself is a response to irrational behaviour affecting a large part of the population.
There’s this weird phenomenon of saying “Haha isn’t it ridiculous that people don’t behave according to utility functions? Now, let’s assume they did…”
Typically the resource in question is money, but he also gives the example of researchers competing for citations and tenure, and grant-writers competing for prestige. I have no doubt that these are things that people compete over, but as a general approach to analyzing social systems, this falls short. What about cooperative endeavours like the open source movement, the occupy movement, or wikipedia? I don’t think the right analysis of those systems starts from competition between agents.
Yudkowsky draws heavily from game theory in the book. And in doing so he arrives at a picture of systems structured almost exclusively by the utility-maximization of the independent agents. We’ve discussed the limitations of utility functions; now we’ll discuss the limitation of independence.
In the world modeled in the book, coordination occurs as a momentary disruption of the normal functioning of the system, and complex interdependence never. For instance, the Visitor describes a system of coordinated action whereby prediction markets and kickstarter-like systems enable large groups of people to all “jump” at once; to take some coordinated action. Notice, though, that this is only an instantaneous deviation from the typical uncoordinated behaviour.
What if there were systems that had interdependence constantly at play? What if the behaviour of some systems is primarily determined by their interconnected nature? Social and business networks are not homogeneous or sparse. There are connectors, mavens, and salesmen who have undue influence, and this structure occurs at all scales. There are also deep interconnections between sectors of the economy, government, religion, and more.
But he does acknowledge interactions between systems. Many of his arguments rest on them. So what am I arguing? Let me explain by analogy. His idea of systems is sort of like simple objects in newtonian mechanics. Can they influence each other in significant ways? Sure. One billiard ball might smack into another, exert some force, and change its trajectory. A rope might constrain the behaviour of the object it’s tied to. This gives us a good framework for discussing billiard balls, mechanical systems, and gasses if we’re clever about it.
But try explaining the processes of the human body exclusively in terms of newtonian mechanics and you’ll quickly find yourself lacking. It’s not that it can’t be done; it’s just the wrong approach. The simplest way to understand the functioning of a human body is biologically, bringing into the foreground the constant complex and dynamic interactions between cells, between organs, with flows of blood, regulatory hormones, and so on. These elements don’t simply act on each other or constrain each other, they are fundamentally intertwined. They must be understood together.
So too, the social systems that comprise our society often have these kinds of complex interdependencies. They shouldn’t be understood as small deviations from an exercise in game theory.
There are a few examples throughout the book of systems that violate these rules. So maybe it’s not a lack of imagination. Maybe there’s an unspoken argument that sure, other systems exist. But the best we’re going to be able to do is equilibrium systems composed of rational self-interested agents competing for some scarce resource in an unstructured arena. So if we show that those systems can be inadequate, even more so anything else! But if this is a premise of the book, he never actually justifies, or even acknowledges it. This is the ideology he expects to share with his reader: that either all systems have those properties, or else that the best ones do.
Another reason he may gravitate towards analyzing these types of systems is that he can. It’s impossible to analyze how any generic social system works, so it’s important to model specific cases. In a sense, this is what Yudkowsky is doing. He is picking a system that his readers are used to thinking about, and showing them how it can lead to systemic inadequacy. It’s important not to forget, though, that you’re only modeling one class of systems.
There are two chapters at the end that I liked quite a bit. One was a takedown of what he calls “blind empiricism”, and which I’ve touched on myself in Why I’m a Platonist. Blind empiricism is the idea that it is better to have no theory at all and work strictly based on crunching data. Yudkowsky identifies the hole in this attitude: that theory is impossible to escape; it is implicit in which data you select. He also correctly describes the scientific process of theorizing and then testing predictions, as opposed to just theorizing or just collecting data.
The second was a psychological profile of sorts. He was describing the cognitive traps that he thinks leads people to an “I can’t do anything better than society” attitude. One was “anxious underconfidence”: underestimating your abilities and only trying what you know you can succeed at. I think I am prone to falling into this trap myself, and I’m working on being more ambitious. I liked the specific way he phrased this:
If you never fail, you’re only trying things that are too easy and playing far below your level.
In other words, you should expect to fail sometimes; never failing is itself a type of meta-failure.
I would recommend this book to anyone that falls inside its target audience. On a personal level, it encourages more ambition, and helps people to identify ways that they can make a difference. And at the level of discourse, it refutes the idea that free markets are inherently good, and that the problems in society are explained by lack of individual responsibility, government regulation, or inherent limitations of reality. It paints a picture of a flawed society that could be improved by systemic changes. I wish it had also questioned economic dogma around the nature of systems themselves, and the supremacy of markets, but then it might not appeal to the people who need to hear this most.
For many, though, it could be a dangerous book. It paints a compelling picture, and unless you are on your guard against the assumptions buried under the text, it is easy to adopt them yourself. Hopefully if you’ve read this review, you’re properly steeled, and can appreciate the valuable ideas that are present. As is often the case, it’s a matter of giving them proper context and knowing when to apply them.