Mondays with the Machine: Thinking Out Loud About the Current "AI" Boom-Bubble
From clay tablets to GPT LLMs: meditating on minds & machines while knowing that "AI" is not your friend but rather your interface to a very weird library. Mental models, moral panics, why prompting is a form of wizardry, and considering MAMLMs as strange mirrors shaped by the patterns of online discourse…
I am going to try to reserve Mondays to think out loud about Modern Advanced Machine Learning Models—MAMLMs. And I am going to try to shut up the rest of the time.
And now it is Monday. So let me start with three observations I think are relevant to MAMLM’s:
The first is an example of the genus of moral panic that happens when advances in information technology shift our distributed brain processing so that more of the current state is stored in front of rather than behind our eyes:
Jim O'Shaughnessy: <https://newsletter.osv.llc/p/on-ai-writing-thought-design-and>: When I was growing up, calculators were new.... And then for a while, they banned calculators from class. Because that was cheating... [for] four years... until people realized... that moral panic was completely misguided.... I think we're... seeing the same thing with schools... banning AI.... It drives me crazy because it's like... "we just discovered how to consistently start a fire."... :You could get burned by that fire, and therefore, you may not do it."... Hey, right after we invented fire, guess what happened? We invented fire alarms, fire departments, firemen, you know, fire exits, all because fire is that powerful. It is incredibly great for us. It's responsible for our prefrontal cortex. Right. Some speculate that it was only when we started cooking our food that the prefrontal cortex emerged…
The calculator is simply one of the most recent of these. After all, in the real old days, you could not be a real scribe unless you spent at least an hour a day getting your clay into the right consistency and then smoothing it, during which time you meditated on what you were going to write that day.
The second is about how the “ChatBot” nature of our interfaces to MAMLMs is misdirecting us by casting a false illusion about what they really are:
Nathan Baschez: <https://newsletter.osv.llc/p/on-ai-writing-thought-design-and>: ‘Because [AI] takes the form... of a chatbot, it feels... like... a person.... [But] AI[s]... really contain multitudes.... A lot depends on the exact way that you tend to prompt it.... Com[ing] from... an engineering background... I'm used to... tinkering with systems until they give me the results that I want. And—rightfully so—that's not the way that we think about collaborating with other people.... If you come to AI with this mindset of... ["]I'm going to cross my arms and see how good of a job you do... from a distance["]... that's just the wrong way to approach it.... To have a tool that... within... two seconds... give[s]... you more choices that you wouldn't have thought of otherwise.... [is an] incredibly valuable thing.... And... it's about learning little tricks... to prompt it...
The way I think about it is this: You are not talking to a person. You are utilizing a natural-language interface to a summarization engine. What does it summarize? The internet, more or less: it tells you what the typical internet s***poster responds when presented with a call like the one you issue.
Why is this useful?
Because if you can poke it properly by your emitting the right calls you can guide it into a corner of the internet where this call-and-response elicits true and accurate information. And because its responses are much easier to parse for interesting tidbits than the ten blue links.
Why are its responses easier to parse?
I guess: (a) Because our minds are built/trained for call-and-response rather than archives. (b) Because its compression of the entire internet into a 3000-dimension vector space is a much finer and more effective classification scheme than <keywords> + <pagerank> or whatever Google and company are really doing these days. (c) This is especially true now that they have been polluted by so much SEO spam tuned to mimic high-quality results for a <keyword> + <pagerank> search.
How can it be so much finer and more effective a classification scheme?
Because of Deep Magic I do not understand. And I need to understand it if I am to have any chance of having even a semi-accurate Visualization of the Cosmic All going forward.
The third is Dan Davies here, talking not about MAMLMs, but rather about learning history:
Dan Davies: <https://backofmind.substack.com/p/the-valve-amplifier-of-history>: ‘“Recognising that an analogy is no good” is a relatively quick cognitive operation…. Problem solving is based on making mental models of the problem; abstracting some of the detail while hoping that you’re capturing enough of the causal structure so that if you solve the problem “in the model” the same solution will work in real life. Usually, disanalogies are quick to spot…. Creating a mental model from scratch is a very expensive cognitive operation, though. So, if you have a supply of previously existing mental models, it might be a very good strategy to just start going through them one by one, effectively running your thumb through the book going “nope, nope, nope, maybe … nope, nope, nope … nah, doesn’t work … maybe … nope, nope … hang on this might work”. Rather than taking on the expensive task of making a model that you’re certain will work because you’ve constructed that way, you’re making multiple cheap attempts. But where might you get a large supply of ready made mental models to go through in this way?…
He goes on:
Dan Davies: ‘Just filling your head up with stories about how things could happen means that you’ve got a catalogue to riffle through, while the constraint that they are stories about things that actually did happen once should exercise some kind of rudimentary quality control on the library of candidate solutions…
The alternatives that MAMLM GPT LLMs offer us in response to our prompts are, it is true, lacking relative to the analogies offered up to us by history. For history, after all—as long as it is true history rather than fantastic, fabulous, or romantic—is “useful to those inquirers who desire an exact knowledge of the past as an aid to the interpretation of the future, which in the course of human things must resemble if it does not reflect it…”
The alternatives that MAMLM GPT LLMs offer us are lacking in that it is unconstrained by thing like, you know, truth and reality.
But they have an edge. They can generate a huge catalogue of alternative possibilities very very quickly. And those are all clustered around the particular point you choose in the 3000-dimension vector space compressed classification of the internet that lies at their heart when they are considered as summarization engines. Thus, if properly prompted, they can get close enough to serve as a place for you to start, or at least to spark thought.
Remaining questions, at least as I see things:
Why are they so good—or are they not so good, but rather a combination of “Clever Hans” and “cold reading”?
What I call “Clever Hans” is when you put as much work into writing the prompt as you would into doing the thing.
What I call “cold reading” is when positive assessments of MAMLMs is based on their having been trained to deliver shotgunning plus Barnum statements, plus confirmation bias.
(Note that shotgunning and Barnum statements can be very useful even when wrong, if they carry other messages, like “this is close to the Python syntax for that”.)
How good can they get, considered as compression-and-summarization-and-alternative-generation engines?
When does the compression-and-summarization engine frame start to mislead, if it ever does?
What will the next thing be in MAMLMs, that does better than training a simulation of the typical internet s***poster?