Taming Our Unaccountable Societal-Scale Machines: How Impossible Is It Going to Be?
“Management Cybernetics” may be the name of the only thing that can save us from our own systems for anthology-intelligence societal-scale coöperation among us East African Plains Apes. But can that discipline exist? It certainly is not really existing now…
Here is one of the many, many eggs I have been juggling—under variable gravity—that I have dropped on the floor in the past six months. It is my promise to myself revise and expand the already 5000-word review I wrote on my last birthday of Dan Davies’s superb little book The Unaccountability Machine <https://www.blackwells.co.uk/bookshop/product/The-Unaccountability-Machine-by-Dan-Davies/9781788169547>:
A taste of what I wrote then:
We have built a world of vast, interlocking systems that no one can fully understands. From corporate behemoths to government bureaucracies, these leviathan-like societal machines with human beings as their parts make decisions that shape our lives—often with disastrous consequences.
Can there be a way to tame these monsters of our own creation, to give them human faces?
Dan Davies thinks the forgotten discipline of “management cybernetics” might provide a way. That is the crux of his brand-new The Unaccountability Machine… [which provides] a much better road towards understanding our current societal-organizational environment than others currently being put forward….
Remember Henry Farrell’s setting the stage….
Human[s]… created a complex world of vast interlocking social and technological mechanisms… impervious to… understanding…. Our first instinct is to populate this incomprehensible new world with gods and demons…”. And then [having done so], Henry says, technoutopianbros or technodystopianbros divide. The utopian… “AI rationalists… [suppose] the purportedly superhuman entities they wish to understand and manipulate are intelligent, reasoning beings… [with] comprehensible goals… [that] might… be trammel[led] through… subtle grammaries of game theory and Bayesian reasoning…. You [may] call spirits from the vasty deep… [and] entrap them in equilibria where they must do what you demand they do…
By contrast, the dystopians believe we should… simply welcome our AI-overlords:
We are confronted by a world of rapid change that will not only defy our efforts to understand it…. We might as well embrace this…. The stars are right, and dark gods are walking backwards from the forthcoming Singularity to remake the past in their image. In one of [Nick] Land’s best known and weirdest quotes: “Machinic desire… rips up political cultures, deletes traditions, dissolves subjectivities, and hacks through security apparatuses, tracking a soulless tropism to zero control…. The history of capitalism is an invasion from the future by an artificial intelligent space that must assemble itself…. Digitocommodification is the index of a cyberpositively escalating technovirus, of the planetary technocapital singularity: a self-organizing insidious traumatism, virtually guiding the entire biological desiring-complex towards post-carbon replicator usurpation…” And this “technocapital singularity” is to be celebrated!…
From my view, of course, Henry Farrell is completely write to judge all this as simply crazy:
For one thing, none of these processes have human intentionality.
We are primed to attribute human intentionality to them for lots of reasons.
But actually believing that they do have human intentionality will lead us far astray.
If these are our other potential guides—and they are—Dan Davies is vastly to be preferred. In the final analysis, therefore, The Unaccountability Machine is our best guide, at least to thought as to how to take action.
But not only have I failed to deliver, I now have to digest as well:
My precis of what they say:
Large AI models should be viewed as cultural and social technologies—definitely not intelligent agents…
They are roughly of the same ilk as past information, communication, and coördination systems like pictures, writing, arithmetic, records, print, video, internet search, markets, bureaucracies, democracies, and ideologies…
Thus we need to ignore fears of AGI and focus on the immediate societal effects of the coming of these cultural and social technologies…
Their ability to do very high-dimensional very big-data regression and classification with an attached natural-language front end makes them powerful tools to generate lossy summaries of massive human-generated data sets—thus transforming very complex assemblies of information into usable forms.
This is analogous to the market system’s summarization of a huge amount of information about production and demand into a single number: a price.
This is analogous to a democracy’s summarization of a huge amount of information about human collective-action goals and constraints into a single decision: a passed parliamentary motion…
This is analogous to a bureaucracy’s summarization of a huge amount of information about past procedures, successes, and failures into a single gate-keeping decision as to whether. for a particular action, established prerequisites have been satisfied…
This is analogous to an ideology’s summarization of a huge amount of information about the world and humanity’s place in it into a single simplified picture of leaders, goals, friends, and enemies…
Users engaging with these systems are not consulting another mind—an intelligent oracle—any more than the market system that directed the boule and demos of the Athenai in the year -456 to source the ten tons of tin for Phidias’s statue of Athene Promakhis from Cornwall was an intelligent being…
Neverthelss, the effects of the introduction of these MAMLMs on society may well rhyme with those of previous transformative cultural-social organizational technologies like movable-type print in the Enlightenment…
The consequences will be the spread of misinformation, bias, and cultural homogenization in some spheres alongside new forms of cultural diversity and immense creativity in others…
There use will uncover non-obvious patterns across human knowledge, generating new avenues for scientific exploration and engineering progress…
They will alter economic power dynamics, as the tension between information producers and distributors will intensify…
The narrative of “AI” as superintelligent agents blocks coherent thought about the immediate and real social and economic opportunities and challenges…
What do I think about this? I do not know yet. I do know that I would like to visit an alternative timeline in which Herbert Simon’s Sciences of the Artificial had become a multidiscipline- and multidepartment-sparking book.
References:
Davies, Dan. 2024. The Unaccountability Machine: Why Big Systems Make Terrible Decisions and How the World Lost Its Mind. London: Profile Books. <https://www.blackwells.co.uk/bookshop/product/The-Unaccountability-Machine-by-Dan-Davies/9781788169547>.
DeLong, J. Bradford. 2024. “A Return of ‘Management Cybernetics’ as a Way Forward Out of Economics-Based Neoliberalism??” Grasping Reality. June 24. <https://braddelong.substack.com/p/a-return-of-management-cybernetics>.
Farrell, Henry. 2024. “Vico’s Singularity”. Programmable Mutter. May 1. <https://www.programmablemutter.com/p/vicos-singularity>.
Farrell, Henry, Alison Gopnik, Cosma Shalizi, and James Evans. 2024. “Large AI Models Are Cultural and Social Technologies.” Science. 387: 6739 (March 13), pp. 1153-6. <https://www.science.org/doi/10.1126/science.adt9819>.
Farrell, Henry. 2024. “Cybernetics Is the Science of the Polycrisis”. Programmable Mutter. April 17. <https://www.programmablemutter.com/p/cybernetics-is-the-science-of-the>.
Simon, Herbert. 1996. The Sciences of the Artificial. 3rd ed. Cambridge, MA: MIT Press. <https://archive.org/details/sciencesofartifi0000simo>.