sourceType:: article author:: Scott Alexander sourcePublication:: Slate Star Codex ref:: https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditations-on-moloch noteTitle:: Slate Star Codex; Meditations on Moloch
Meditations on Moloch
2021-11-26
Scott Alexander
https://www.lesswrong.com/posts/TxcRbCYHaeL59aY7E/meditations-on-moloch or https://slatestarcodex.com/2014/07/30/meditations-on-moloch/
Like the rats, who gradually lose all values except sheer competition, so companies in an economic environment of sufficiently intense competition are forced to abandon all values except optimizing-for-profit or else be outcompeted by companies that optimized for profit better and so can sell the same service at a lower price.
Whether or not it was a mistake, it wasn’t an accident – agricultural civilizations simply outcompeted nomadic ones, inevitable and irresistably. Classic Malthusian trap
Races to the bottom:
Once one agent learns how to become more competitive by sacrificing a common value, all its competitors must also sacrifice that value or be outcompeted and replaced by the less scrupulous. Therefore, the system is likely to end up with everyone once again equally competitive, but the sacrificed value is gone forever. From a god’s-eye-view, the competitors know they will all be worse off if they defect, but from within the system, given insufficient coordination it’s impossible to avoid
Multipolar traps:
In some competition optimizing for X, the opportunity arises to throw some other value under the bus for improved X. Those who take it prosper. Those who don’t take it die out. Eventually, everyone’s relative status is about the same as before, but everyone’s absolute status is worse than before. The process continues until all other values that can be traded off have been – in other words, until human ingenuity cannot possibly figure out a way to make things any worse.
The reason our current system isn’t a utopia is that it wasn’t designed by humans. Just as you can look at an arid terrain and determine what shape a river will one day take by assuming water will obey gravity, so you can look at a civilization and determine what shape its institutions will one day take by assuming people will obey incentives.
Vegas
by what standard is building gigantic forty-story-high indoor replicas of Venice, Paris, Rome, Egypt, and Camelot side-by-side, filled with albino tigers, in the middle of the most inhospitable desert in North America, a remotely sane use of our civilization’s limited resources?
And it occurred to me that maybe there is no philosophy on Earth that would endorse the existence of Las Vegas. Even Objectivism, which is usually my go-to philosophy for justifying the excesses of capitalism, at least grounds it in the belief that capitalism improves people’s lives. Henry Ford was virtuous because he allowed lots of otherwise car-less people to obtain cars and so made them better off. What does Vegas do? Promise a bunch of shmucks free money and not give it to them.
But not only have we not yet reached the sea, but we also seem to move uphill surprisingly often. Why do things not degenerate more and more until we are back at subsistence level? I can think of three bad reasons – excess resources, physical limitations, and utility maximization – plus one good reason – coordination.
Reasons for above:
- Excess resources (for the moment, in some places)
- Physical limitations (slaves aren’t whipped more not because of morals, but because they’ll die. We don’t overpopulate like the Malthusian rats because of gestation time)
- Utility maximization
- Coordination
one of the reasons we’re not currently in a Malthusian population explosion right now is that women can only have one baby per nine months. If those weird religious sects that demand their members have as many babies as possible could copy-paste themselves, we would be in really bad shape. As it is they can only do a small amount of damage per generation.
In a sense, evolution solved for that with 9 moth gestation
The coffee plantation needs to make good coffee so Americans will buy it, needs to treat employees well so people work there but that’s all, and only as much as necessary for the given context (you can lower wages if there’s a job shortage)
Maybe they’re on the ancestral burial ground of a tribe different from the one the plantation is employing, and they want it respected in some way. Maybe coffee growing contributes to global warming somehow. As long as it’s not a value that will prevent the average American from buying from them or the average Ethiopian from working for them, under the bus it goes.
in theory it’s optimizing for voter happiness which correlates with good policymaking. But as soon as there’s the slightest disconnect between good policymaking and electability, good policymaking has to get thrown under the bus.
Becoming a super organism would allow us to take a gods eye view and solve meta problems. Government-ish.
The best known solution to the Prisoners’ Dilemma is for the mob boss (playing the role of a governor) to threaten to shoot any prisoner who defects. The solution to companies polluting and harming workers is government regulations against such. Governments solve arm races within a country by maintaining a monopoly on the use of force,
Social codes, gentlemens’ agreements, industrial guilds, criminal organizations, traditions, friendships, schools, corporations, and religions are all coordinating institutions that keep us out of traps by changing our incentives.
The libertarian-authoritarian axis on the Political Compass is a tradeoff between discoordination and tyranny. You can have everything perfectly coordinated by someone with a god’s-eye-view – but then you risk Stalin. And you can be totally free of all central authority – but then you’re stuck in every stupid multipolar trap Moloch can devise
Tangent: becoming one. If you imagine an AI that copies itself many times you don’t think about it competing with itself. Even though there may be very many agents taking over the world they don’t seem to be in competition with each other – because they are all one, in a sense.
Technology (as a stand in for time, as “advancer”)