I’ve been remarkably quiet about AI on this newsletter. I think it is time to change that.
Upfront, this is a long one. Enjoy.
I conducted a search on all the writing I have done on this newsletter about AI since its inception (not other places I write, only here) and I have mentioned AI 17 times. I first mentioned in 2019, where I naively discussed the generalisation of AI for automation, which, I essentially, argued that manufacturing robots and automated lines would be boosted through AI to make things more efficient. A full two years before the outing of ChatGPT. Naively, because I fell into the trap that so many people do currently by not defining ‘which’ type of AI they are talking about. Today, the term is used in a dangerously reductive form, suggesting to many that all AI is chatbot Large Language Models (LLM). I was, of course, talking specifically about Machine Learning (ML) at the time, but you’ll have to believe me. I was aware of GPT models from around 2017/2018, where they had been used by malware actors to ‘mimic’ text in emails to avoid anti-spam detection and target specific users using classic social engineering. But they were nowhere near as sophisticated back then.
Since then, I have discussed AI in terms of use on so-called Big Data, taking care to discuss the downsides and risks of using data in too much of a cavalier fashion. I saw demonstrations during a conference where ML was successfully used to help organise masses of invoices imported from PDF copies. Clearly a great demo, but how many businesses are faced with that situation? Typically, businesses will need to digest data on an ongoing basis (as backups serve the purpose of restoration in the event of catastrophe), but the savings in time are small in the instant, although they add up over a year or so. Obviously, the pandemic turbocharged the idea of using AI and I discussed that a little, and presented to a wide audience some of the trends I had been noticing that would affect the Caribbean as a whole during a conference.
In September 2020, I rightly observed that NVIDIA was about to go from being a well-known PC graphics card company to being the backbone of everything AI, by a freak of luck in how mathematics works. Side note: I was talking about ARM a lot, too, and that it was headed to greater things in computing hardware —five years later and the benefits of the architecture are apparent. Subsequently, I have only in passing, mentioned how AI (again, I poorly defined what I was talking about) was going to become more and more mainstream, but it was last on a list of things like Cloud Computing, Digital Transformation, Regulation, Security, and Mis/Dis-information! If I had to try to explain, I would suggest that, even back then, whilst I saw some use for it, I was much more sceptical in its transformative capacity than most.
In Computational, I discussed how the robots were going to take over, but not in the way you think. Then in February, I relayed some feelings and responses that I had been seeing first hand from training sessions I had been giving for local businesses interested in the technology. Even then, I was circumspect about the universal usefulness and the risks of using it. I wrote:
It is clear to me that I have been surprised by the interest from such a broad range of managers and business leaders for a product that is so technical and so linked to ICT. The OpenAI hype machine has galvanised the public into believing that these tools can make them one hundred or more times as efficient for 100 times less money than they are spending at the moment (on personnel). This, of course, is not true at all, and I find I have to temper expectations and canalise those runaway thoughts they often have about generative AI and how it will make every person redundant.
I don’t think discussing accuracy, efficiency or other measures of “intelligence” is helpful at this stage, as these systems are changing rapidly. To give you an example, I have had to modify the training materials no less than ten times in the last six months. I would suggest a wait-and-see approach before integrating them into fundamental or central processes in your businesses that would provoke significant consequences in the case of error or failure. I would also suggest you integrate human-based verification and validation to the output generated to ensure you don’t fall foul of mis and dis-information, obviously wrong answers, and poor analysis that these LLMs can produce. That doesn’t mean that I don’t support the use of them. Please do. However, please don’t rely on them too much, as you may be sorely disappointed and dissatisfied with the results.
This is just to lay the ground about how I currently fell about LLMs and AI in general.
Not all AI is the same
Generally, I think it is important to be clear about what it is you’re talking about. If two or more people are ostensibly talking about the same subject, then it is imperative that all parties are using a common working definition. This is often not the case when discussing AI, and this failing is actively exploited by firms and individuals for their gain. For example, many people like to discuss “freedom of speech”, particularly in an online context. But what most ignore, is that that definition is not the same for the entire world. The European Court of Human Rights and the European Convention on Human Rights changed their definition from “Freedom of speech” to “Freedom of expression”. Notice the difference. Speech was too limited in scope, in that an act was not counted. With freedom of expression and act (symbolic speech), is included. When you ‘like’ something on the Internet, you’re not speaking, but your actions are a form of expression that you have made public. Of course, we can get into all sorts of debates about what the ‘like’ button actually means and the “correct” meaning of freedom, etc, but that’s not the object of this discussion.
When talking about AI, what people seem to be talking about is LLM chatbots. This reductive use of the term is dangerous, and it lets the firms that want to control our collective experience of the Internet dictate the terms and use of the definition. It is dangerous because those uninitiated in the technology might think that a system that was fully automated and reliably accurate was run on ChatGPT. Perhaps encouraging them to invest to implement tools in their processes, only to find that for some odd reason their system doesn’t work so well. Have you spotted what’s wrong? Of course, an LLM is a statistical next-word generator and a Machine Learning (ML) algorithm is something entirely different with behaviours and outputs that cannot be compared with an LLM. This example should also make you think about the ultimate use of the outcomes. What if, for example, a dataset was being used to determine whether a person receives a live-saving benefit or not, ultimately determining if the person will live or die. Is it acceptable to misuse the term AI, then?
Where do I stand on LLMs?
Off the bat, I should be clear about my feelings on the technology. Separating everything, and looking purely at the technological prowess of LLMs, it is clear to me that there are some advances that are significant and that certain use cases in processing natural language, for example, may prove useful.
This statement, you should notice, is not particularly a full-on endorsement, nor a definitive and tranchant position. Why is one’s cul entre deux chaises?
For me, it comes down to several issues that, on balance, feel deeply problematic for me. Unless you’ve been disconnected from the Internet and hidden yourselves from international news, it should come as no surprise that LLM technology has been used for the most revolting and criminal behaviours we have seen, and slowly integrating itself in military applications, such as aiding a full on genocide in Gaza. Is it the fault of the LLMs themselves? Well, in part, absolutely. And, more specifically, it is the fault of the engineers and the managers who develop these systems as much as it is the fault of the end users who willingly partake in the production of these outputs.
I feel like I shouldn’t need to go into this in detail, but every so often I wonder. Briefly, an LLM is the product of its ingested and calculated material. The very fact that the ingress data is as biased, misrepresentative of the world, and rotten, it should be no surprise that the egress would be anything but similar. The fact that this is frequently denied, misdiagnosed, misinterpreted and misrepresented should tell enough about what you should think of the companies developing these systems. You, alone, should decide where you fit on that scale.
I, myself, am deeply troubled by the cavalier attitude to the use and misuse of data that in some cases is open (i.e. free to access, use, and reuse), but in numerous instances is clearly protected material that has been misappropriated and has no place in the training datasets being used by the LLM-makers without consent.
But I think the most egregious ‘fault’ of the LLM-makers is their consistent false narratives about efficacy, productivity and probity of what are, provably, just next-word-prediction-calculators. Am I saying there is no use, and they are a waste of time? Please re-read the first paragraph in this section.
The way LLM-makers aggressively push their services with demonstrably false claims is nothing short of disgraceful. They have won over the public with two strategies, that I think they chanced upon, rather than plotting world domination like “The Brain”. The first is the age-old trick of the door-to-door salesmen used to sell you crap, that at first glance looks fucking amazeballs, but in subsequent use, you realise that you’ve been had. The first time you type a request into a chatbot, and it blurts a bunch of text about a subject that you’re interested in, it seems quite impressive, and, to be fair, the prose is not terribly written —for that see my writing.😃 But on closer inspection, particularly if you are an expert in the subject, you start to notice areas of supposition, lack of detail, poor factual datapoints, and often, invented information that doesn’t exist. I recently read a quote that went something like this:
It’s weird. When I ask ChatGPT to tell me something about a topic I don’t know much about, it is really good and shows a level of understanding that is impressive. But when I ask it something about a subject I am an expert in, I find so many mistakes that it is not always that useful.
If you thought Dunning-Kruger reading that quote, you pass Go and Collect €200.
I’ll give you an example, one that I use in my training. We ask the LLM of choice to explain why there are so many traffic problems in Martinique. The answers look convincing and are plausible at first glance. And, not dissimilar to a stopped clock being correct twice a day, they even hit upon the ‘real’ reasons from time to time. On inspection, however, they frequently offer up stereotypical (and, just shy of prejudiced) explanations. From them being the fault of tourists creating traffic jams (laughably implausible), to offering up that the islands are mountainous (true) with poor roads in these parts (true) and that contributes. For clarification, there are no regular traffic jams in the mountains. But, as mentioned, there are explanations, such as rush hour, a concentration of economic activity, poor public transport, and others. Yes! It got it right. Oh! What? That’s true for pretty much any place in the world where the conditions align, did you say? Like every town in the world?
The other way the LLM-makers try to exercise control over the narrative, leading you to believe their immense and super human powers, is by a simple technique of anthropomorphisation.1 As humans, we’re hardwired to see and feel human-like qualities in things. Just as we see a face in the frontend of a car, we see and feel an intense emotion to believe what is written in front of us is human-like. Coupled with our own achilles heel of confirmation bias (a phenomenon that we’re all vulnerable to) we can easily fall into the trap of trust in the outputs.2 ELIZA was the first real-world example of this, and today’s LLMs are no different.
But there is a difference today. The LLM-makers are actively exploiting that human weakness. They are deliberately using anthropomorphising terminology when talking about these statistical next-word calculators. They use terms like “thinking”, “he/she”, “feels”, “understands” among others. These are conjurer’s tricks to avoid disbelief of the act being performed. They are also very dangerous. The sleight of hand being performed, at its heart, is to displace responsibility from the humans creating and operating these systems onto the computer program, as if a computer has rights and, more importantly, responsibilities like humans do. The desired outcome is to ensure that they are never held accountable for horrific consequences. Like the suicide of a teenager, who, on advice and encouragement from ChatGPT.3 See also the deepfake child pornography site formerly known as Twitter.
It is not new either. This from 1976:
If a researcher … calls the main loop of his program “UNDERSTAND,” he is (until proven innocent) merely begging the question. He may mislead a lot of people, most prominently himself. … What he should do instead is refer to this main loop as “G0034,” and see if he can convince himself or anyone else that G0034 implements some part of understanding. … Many instructive examples of wishful mnemonics by AI researchers come to mind once you see the point.
My source: https://www.techpolicy.press/we-need-to-talk-about-how-we-talk-about-ai/
Then we arrive at a highly disputed topic concerning the development and operation of LLMs, that of the power and cooling necessary to run huge GPU farms destined for training and operating to reply to your frivolous little chat to find a recipe for tonight’s supper.
Currently, the debate about the power use of LLM-makers is either presented as something that is accelerating the destruction of the planet due to the burning of evermore fossil fuels to power datacenters, to a use that is, in the grand scheme of things, inconsequential. This is a highly disputed topic and one that I don’t have clear answers for, but there have been a number of reports and studies that show that these systems do indeed use a lot more energy than previously estimated. I know there are plenty of reports bounding around that show that the LLMs only use marginal amounts of electricity and cooling and that we shouldn’t worry our pretty little heads with that and carry talking to our digital mistresses and making unsavoury content on the previously mentioned deepfake child pornography site.
In 1964, the Surgeon General of the United States of America released the first report on smoking and health. It concluded that smoking was:
- A cause of lung cancer and laryngeal cancer in men
- A probable cause of lung cancer in women
- The most important cause of chronic bronchitis
But this was after the tobacco industry had released their report, in 1954 entitled “A Frank Statement to Cigarette Smokers” whereby they proceeded to tell people that smoking was not detrimental to health. The tobacco companies all ‘knew’ that this was a lie, but they kept peddling it until the lie was untenable and enough evidence had accumulated that they finally caved sometime around 1999. (Read that date again).
I believe what we are witnessing with the LLM-makers is the same dynamic being played out, albeit at 2020s speeds. I believe the LLM-makers are fully aware and are deliberately holding off granting full and unbridled access and analysis because it will be shown to be worse than most people think. I believe they are doing so because they have started to enjoy the smell of their own farts about a statistical next-word guesser ‘finding and innovative solution to the world’s energy issues’. It is delusional, and in saner times would have most of us recommending psychiatric treatment to these people.
Possibly the biggest issue of all, however, is that it seems the amount of money required to keep the lights on currently —and required in the near future— cannot be sustained to the point at which a bubble (multiple bubbles?) will burst faster than on a silly TV game show.
This essay is already very long, so I’ll spare you the excruciating detail, there are plenty of others who have and have done a much better job than I could.4 Suffice to say that the money required to build a datacenter, plus the costs of training the models, added to the costs of running the applications for each query far outweighs the revenues, to a point that multiple tens of billions of dollars are being lost in the industry per quarter and there is absolutely no path to profit for the foreseeable future.
Add to that the development of free and open models from the competition that effectively reduce the ‘value’ of an LLM to zero, and that makes the situation much harder for the hyperscalers. Finance bros and tech bros like to talk about “moats”, ignorant that they weren’t the great defences they are cracked up to be. In other words, why would you pay, if you could get access to something similar for free? Probably jingoistic tendencies, if we’re honest, but that won’t be enough of a population to help turn a healthy profit with today’s systems. The confluence of the previously discussed inefficacy and mild productivity gains in certain circumstances would lead us to understand that something has to correct or reset. Not the boosters, though. Oh, no. If anything, they say it “proves” that they need more money, not less. Like a homeopathy peddler convinced that even less of the active ingredient makes the medicine stronger. (I use the term medicine lightly here, as I’m sure you’re aware).
What have I done, and what am I doing going forward?
Well, those are two different questions of course, but if I were to summarise, it’s a combination of limited use, mindful language when discussing these systems, and taking care and time to train people on the regulatory implications, policy, as well as the negative aspects of LLM use to help provide a fair and balanced understanding of a toolset that has for now become imbedded in today’s computer use.
As far as use goes. I very rarely use them, and I never use the outputs directly in any work I produce. I have and will use them now and again for ideation, or to get the “motor kickstarted”, where I sometimes find them helpful to combat procrastination and other difficulties. Sometimes, longer form outputs can be helpful for structure, ideas and general pointers. I’ve recently used Claude to help me resolve a couple of formatting issues in the .css on a Hugo-based system. They were helpful, but not the panacea the vibe-coding fraternity will have you believe. Leading me in circles at times, but giving me enough of a pointer to find real help that actually resolved the issue. Note: I use grammar-checkers that are increasingly “AI-powered”, so it is difficult to totally avoid LLM use when writing.
When I’m training, I take time to explain how LLMs work in simple terms then quickly move on the uses and abuses of the LLMS, the back end excesses, the atrocious conditions of the moderators, the risks of data privacy breaches, the risks of being had by the machine and I take time to stress that any use of the system unequivocally requires responsibility and expertise. If you abdicate your responsibility over to an LLM, you will regret it. As experts in a given field or subject, you should maintain expertise and only use the systems for assistance. If you are not an expert in a topic, and you rely too heavily on the output of an LLM, you are playing with fire, as plenty of legal personnel have found to their detriment. There have been plenty of fabricated legal texts submitted through the use of LLMs, in Turkey, Austria, Brazil, the USA, Czech Republic, Argentina, …need I list them all?5 I confront the racialised and racist outputs generated, and give advice on how to spot them, as well as explaining how that happens —hint: 💩in = 💩out.
I don’t know what the LLMs will eventually become, but I do know that the current mindset of the devs and operators, including the tech bros behind them are not the right path, and it is something that requires correction soon.
One last point I’d like to include here, and something that merits more discussion in the future, is that a cult belief has recently been born out of fear of a bubble bursting, that it doesn’t matter if there’s a crash because it’ll leave all this lovely datacenter infrastructure available for future innovation, just like the fiber deployments in the USA did after the dot-com crash of 2002. (Not true). TBC.
Thanks for reading.
Have a most excellent week.
