Blog

The Slopocene

I think it’s safe to say that humanity has officially entered the slopocene.

The hype machine is in full swing. Almost daily new stories are published about how Large Language Models (LLMs, “generative AI”) are going to replace human labour. Vibe coders have fully joined the scene pumping out huge volumes of slopcode. Large corporate entities have started ripping off each other’s open-source software by rebuilding it from the ground up, using the suite of tests used to check the code is working properly to do so. Independent developers have even started reverse engineering proprietary software to provide a slop Adobe Creative Suite. People are increasingly turning to chatbots to do research. It’s easier to sift through yesterday’s regurgitated leftovers if it’s interactive and most search results are overrun by articles written by chatbots anyway. Almost every service you interact with is so excited to tell you about its new ai-assisted features.

AI musicians have made it into the charts. AI influencers are, well, influencing people. It’s being used to generate political slopaganda, and genuine news stories are oft accused of being slop. The debate around Artificial General Intelligence (AGI, frequently conflated with machine sentience) is still active, though is shifting toward whether we’re building Skynet and collectively writing our own death warrants. It seems to have a habit of coming up in every single conversation you have whether you like it or not. It’s inescapable.

Meanwhile the verdict is still out as to whether this is a new industrial revolution or a second 2008 financial crisis. Adoption has rapidly accelerated across the industry this year, the rise and fall of “tokenmaxxing” (company incentivised usage of LLMs and AI agents to automate workflows) has mostly passed. Bills are coming due on initial investments, and Anthropic has filed to sell shares on the public market. Usage is skyrocketing, but the actual ability of its usage to return an unambiguous profit is yet another debate.

What does this mean? Where is it going? Will the LLM bubble have popped before I finish writing this?

I don’t know, but I’m ready to have a guess. Apart from when the bubble pops, I’m not a betting man – though I’m sure it will.

There are only two futures for the current hype cycle that I really feel are worth discussing. One is if the hype bears out; the other is if the bubble bursts. By ‘burst’, I don’t necessarily mean a tidal wave of financial carnage, though that’s certainly possible, but I think that it is fairly uncontroversial to point out that a financial correction is very likely on the way or has already started.

I’ve indicated where I think this is going, so let’s start by looking at the other path.

As I am convinced that this path is a fiction, I present a short story to outline problems I don’t think we are prepared for should some of the crazier fantasies become reality.

The Desert Of The Real

It’s 2032.

OpenAI achieved Artificial General Intelligence five years ago.

2027:
The jobs crisis peaks, unemployment soars, AI companies represent a significant portion of the stock market. LLMs replacing human labour is largely responsible for the situation.

2028:
The achievement of AGI was ratified by a panel consisting of Geoffrey Hinton, Richard Dawkins, Karen Hao, Ed Zitron, Corey Doctorow and Elon Musk. They unanimously deemed GPT-7 capable of reasoning on par with a human. The company has petitioned the US government to confer the legal status of artificial person to the model. Doing so would give the company a legal obligation to keep GPT-7’s systems powered and running indefinitely or be charged with murder. The relationship between Microsoft and OpenAI is officially ended.

2029:
It is no longer possible to distinguish artificial “fake” media from the real thing. Digital information is no longer considered trustworthy unless directly transmitted between trusted parties or word of mouth. Reliable digital communications are only possible with people you have physically exchanged keys with. Anything not cryptographically signed is too easy to fake. Automated infrastructure delivers most digital goods and services.

2030:
The government barely exists. True power is exerted by a corporatocracy of service providers. With reality being impossible to verify outside of your immediate surroundings, politics becomes little more than reality TV. What’s real doesn’t really matter as long as the general population is provided for.

2031:
In a world with no labour there is a surprising dearth of leisure. Vocations are rare and previously skilled specialists are disappearing through atrophy. In person interaction fades and relationships that are easier to maintain, relationships with AI companions become commonplace. There is nothing to work toward or against.

2032:
Some people know who’s running the world these days, but not many. Societal inequality rivals that which was ended by the printing press. Most exist as little more than cattle. They want for nothing, their minds gently coerced by an unending stream of content individually tailored to keep each docile. Occasionally you see people who seem to know what is going on but you never meet them. You can barely tell the difference between reality and your content feed. You hear people have started uploading their consciousness in order to live forever. You don’t think you could tell if they’d done it to you already or not.

There is no social contract. There is no society. There is just an endless desert of consumption, an ouroboros of regurgitate.

Dystopia Now

Of course, I’m not a prophet. Maybe AI really does revolutionise everything and we end up living in a post-work fully automated luxury communist dream society. However I don’t think we’re prepared at any level for that eventuality, and anyone who thinks we are lives in a pretty big ivory tower.

Every truly utopian vision of a future with AI relies on either a benevolent dictator or equity of ownership and control. We have neither. I would argue that accommodating any kind of dictator is ceding too much control; especially when concerning the future of truth itself. My little story may have been hyperbolic, but the ramifications aren’t. Ownership of the infrastructure is a problem now. Its impact on society is a problem now. Its impact on our self expression is a problem now.

Increasingly we are excluded from control of our own lives or how they are governed. In an age of media driven by social algorithms this is becoming a problem for our private leisure too. When creatives are forced to compete with slop, their output trends toward slop too. I’ve already written about how the structure created by this system rewards trends toward the homogenised and/or derivative.

This scenario creates an environment that pushes people toward low effort, low quality work that erodes skill. It also floods the zone whilst maintaining audience capture. It creates a vicious feedback loop of “if you can’t beat them, join them”. Why put effort into something when people are rewarded for laziness? People using algorithms to create material to appeal to algorithms that are designed to make people act more legibly to algorithms.

These forces already act on us, parts of this dystopia are already quite real.

The construction, training and running of these massive frontier systems is ultimately an unprecedented siphoning of collectively produced public information, provided for free at point of access, into private hands. It feels like an attempt to enshittify the institutions of learning themselves. Of course the original information is still out there on the internet for the most part, but the success of LLMs disincentivises those who innovate without, and blinkers those who create with. A statistical model trends toward preexisting patterns – they exist explicitly to identify them. This pattern predates LLMs, but they put booster rockets on this pattern, something else I’ve also written about at length.

So why do I think it’s so likely that the bubble will pop? I think it’s worth addressing before looking at what might happen afterwards.

The Singularitopalypse

One thing the hype cycle has relied on historically is the narrative on continuous growth: that the models will just keep getting smarter, that we just need to keep training better models. Even the shouting about a ‘pause’ feels like it’s couched in the dynamics of a Mexican standoff. Like a nuclear non-proliferation treaty the words are cheap to say, but harder to act on – and rely on the trust that the other parties will actually stop. These companies owe their future to having an edge. All the good will in the world won’t stop them trying to retain it. But this narrative of growth has a central problem that has slowly become louder over the last few years: how is this achieved?

Companies started with a massive corpus of questionably obtained information from the public internet – a massive amount of training data. They mined large communities of open source code (especially GitHub). Now they use the data you feed them whilst interacting with them, and use a combination of techniques to attempt to refine the information they already have. They may assure you that your sessions are private, but you only need to look at how surveillance capitalism is used with ‘anonymous’ data to understand how these rules can be bent and broken. Even if they say in their legals they don’t use what you work on for training, I wouldn’t trust it.

The issue is that, as with many areas, the 80/20 rule seems to ring true. The initial progress is comparatively easy to achieve, but over time the amount of effort required for continued improvement gets progressively harder. The free easy data to obtain has already been mined, by this point (discounting destroying books to continue training). The amount of training compute to create these improvements has been growing by a factor of around four to five year on year since 2020. Synthetic data – that is, LLM output created for use as new training input – has been one answer. These refinement and recursive techniques can have real utility, but without careful utilisation the risk is that training models on slop can lead to a form of “AI Alzheimer’s” called model collapse.

The upshot of all of this comes back to the 80/20 rule. Progress is getting harder to pursue and only more so over time. The narrative of the AI bubble and the singularity – where AI outperforms people on every level – is cut from a similar cloth in terms of the conversation around inevitability. And both arguments tend to have a similar public perception – that of the extreme fantasist.

Even if AI superiority has taken a back seat in recent months, the industry sprang from this vision. Major players still believe it, and a lot of money is riding on it. And even if the systems that have been created as a result have some utility, there are some major issues with the financial model being used to drive it. One that is a major threat to the companies building these absolutely massive, expensive models. Infinite growth is never guaranteed, infinite change, at least until the heat death of the universe, is assured.

Sloponomics

Everything that has taken us to this point has required massive amounts of infrastructure spending and financial investment. At time of writing, AI-related spending across the economy is projected to total $2.7 trillion by the end of the year. Market penetration is, as previously discussed, incredible. You can’t get away from something being presented to you via an LLM in some way. It is one of the most rapidly adopted technologies of all time – faster than the internet or the personal computer. These arguments are published online with breathless reverence, but it’s worth looking a little closer at what this means for the AI economy. The dominant narrative is of a hyperjump into the future. A new frontier in ascendance, a second industrial revolution, a reinvention of the capability of humanity.

I think it’s important to take into account what the ‘AI economy’ actually is, how it interacts with industry and how we choose to measure that impact.

I am not about to argue that people “don’t really use AI” or that it has no utility. In an exhausted, time-strapped world, these language engines have become a truly useful way of automating busywork. But part of the reason for such massive penetration is how unavoidable using it is – and free at the point of use. The top use cases I see for the consumer are effectively as an alternative to a search engine, a counsellor/confidant, or an automated word processor. Tasks that aren’t hard but, like with most SaaS platforms, are aided by this extra layer. But the companies that run, host and build these models have real bills that need paying, and are really struggling with profitability of frontier AI.

Amazon is desperately buying so many GPUs that it can’t turn them all on, hardware that is notorious for rapid depreciation as new generations are manufactured. Until the data centre construction and power demand complications are resolved, none of it can make money for them. The finances aren’t looking great. Not to mention the circular financing arrangements that appear to be rife across the industry. The question becomes “when will some money enter from outside the system?”.

As mentioned, a large portion of the end users of LLM providers don’t pay for them. If they do they’re bundled as an add-on for a pre-existing service. A significant portion of the hyperscaler’s (Google, Meta, Microsoft, etc) AI-related income comes from other AI companies using their resources. Moreover, the risk concentration in the US stock market greatly weighs on these companies, with them taking a large share of responsibility for current trends. This shelters the end user from the true cost of the service they’re using, and many of them are unlikely to pay for something that, in general, has a mild impact on their day – casually or professionally. It’s potentially a lot of embedded chatbots suddenly not justifying their cost.

Combine this with increasing pressure on companies like OpenAI and Anthropic to turn real profits that come from outside of the industry and the outlook doesn’t look fantastic.

The ubiquity of LLMs isn’t necessarily an indicator of their indispensability. Neither is their utility. An ice cream machine can be useful, but price can make owning one unjustifiable. So-called ‘narrow’ models trained for specialist tasks (for example, Microsoft’s Phi-4 model, a ‘Small Language Model’ geared toward reasoning and mathematics – more on that later) can be good at specific jobs. This is a lot closer to how classical machine learning is used, but these small models won’t command the kind of revenues needed to justify the continued operation of these companies in this way.

And when(?) the bubble pops, these forces won’t disappear overnight. Pandora’s box has been opened.

Once You Pop You Can’t Just Stop

As the finances start to wobble, as they already seem to be doing in China, I expect the first consequence to be a tightening of training budgets. Distillation, synthetic data and specialisation will all get a higher focus. Microsoft’s Phi-4 is a good example of this: a model trained on carefully curated mixes of synthetic and ‘organic’ data with the goal of providing greater reasoning and problem-solving capabilities with a smaller footprint. Synthetic data has proven especially useful for this, but the kinds of generalist intelligence we’ve been taught to expect from ChatGPT are the ones that are particularly dependent on diverse inputs. Repeated training on LLM outputs risks model collapse, and even with models trained mainly on synthetic data there still seems to be a general level of reliance on ‘organic’ human-created material.

On the consumer end, the very first thing that will (and has already started to) happen is that LLMs will get more expensive to use, especially for the flagship models. There is good and bad in this. On the positive side, many redundancies will need to be reversed. The collapse in demand for high-quality computer parts is likely to reduce the price of consumer electronics, games consoles and PCs. A welcome change indeed considering the direction the market is currently headed. Also it hopefully means that made-up skills that no one can quantify become less important on CVs, as companies run out of money to throw at Synthetic Text Extruders (a more accurate description of an LLM) and can’t give their employees unlimited free access. Sanity might start to prevail somewhat. I’m sure that, as costs go up, “prompt engineering” – the art of knowing how to whisper to an LLM in the right way to get it to do what you want – will become a more valued skill, but in a narrow niche.

As prices go up, people will be looking to save costs and focus on smaller, more efficient models. There will be a focus on self-hosting setups that are cheaper to run, which will also likely create a convenient demand for more powerful consumer electronics. Whether this will be justification enough to keep all prices high remains to be seen. It would, of course, depend on the perceived value of a smaller, less powerful version of something you used to get for free. My guess would be that specialising these models for more narrowly defined tasks (so-called “thin” AI) will give them a veneer of improvement to make them desirable. How fit for purpose this will be will come down to use case. More innocuous, natural language processing stuff could work quite well. Therapy chatbots and medical diagnosis apps will likely be very damaging.

Now the models are real things that exist, they are unlikely to go away. Expensive to use and limited access maybe, but if they provide real tangible benefits then they will be used by people with the resources to do so. Economically, I’m not certain the gains will be completely justified for most applications. I’m sure there’s a decent chance that a company like Palantir might survive. Flimsily justified murder feels like something you can make budgetary justifications for as a government. Usage as human decision accelerators is probably the most damaging application they can be put to. It provides ample opportunity for systemically ingrained bias to tip the scales in situations where the system often wants you to make unethical decisions to save money. Things like denying healthcare and insurance claims, access to benefits, legal decisions. I can see a world where these LLM services are used by the largest private entities to grease bureaucratic wheels using data science. The question really becomes how far into the weeds does this technology retreat? How do you avoid being stuck on an automated customer service call that never ends, where there is never a human to talk to on the other end? YouTube is already there. Balancing the equation of value to cost for these services is going to have a massive influence on the way this paradigm shifts.

Speaking of social media, I don’t think we’ve reached the end of the slopocene just yet. Attention farming and faked propaganda media are plenty valuable. I wouldn’t be surprised if nefarious organisations and / or governments manage to make the space on their balance sheets to pay for premium slop generation.

Sloppy Seconds

For years we’ve been told that “AI is coming to take our jobs”. It has become a meme:

“I want AI to do my laundry and dishes so that I can do art and writing, not for AI to do my art and writing so that I can do my laundry and dishes.” @AuthorJMac

The intended goal from the beginning of this cycle was to make human labour redundant, or, if that isn’t possible, to at least markedly reduce its value. Benn Jordan calls this “Leveragism”: the combined effort to reduce the value of everyone’s paycheque just enough to keep the majority in the rat race, fighting to tread water and keep their stability. Too exhausted or too scared to resist or make meaningful change. These companies also invest huge amounts in lobbying funds, making it difficult to resist in a democratic way.

They want us poor. They need us poor. I would look very carefully at those opposing wealth taxes and what their motivations may be. Musk recently, briefly, was the world’s first trillionaire.

Whilst RAM, disk storage, and hardware prices are likely to go down after the bubble pops, the decline will, in all likelihood, be anything but instant. If suppliers have thought ahead, they may well artificially restrict production or build expensive new infrastructure to keep prices high.

Worldwide, insane amounts of cash are being funnelled into constructing data centres. If demand were to suddenly collapse, a lot of people would find they weren’t getting the revenue they expected. With people invested so deeply, it wouldn’t be that surprising if it triggered a new 2008-style recession.

In a way, it feels like corporate finance is trying to destroy the state. Destroying the economy will likely cause even more privatisation, as public funds get slashed to support a crumbling system. If inequality isn’t addressed as a matter of urgency, then it’s quite possible this kind of “leveragism” ends in a modern equivalent of feudalism. The Wayland-Yutani style future that Alien presents us with: a world where people are owned by corporations and work to pay off their debt. My personal theory is that AI companies are trying to scare governments into nationalising them. Doing so would give the current owners a way out, leaving a huge amount of debt on the shoulders of public institutions. There are details in Anthropic’s IPO filings that allow existing investors to cash out. Their own filings disclose around $900 billion they’ve committed to spending on compute and infrastructure, much of it years before it becomes operational. Some of that may be conditional, but these are very large numbers.

The public could be left holding the bag, paying for it with tax money, pushing the state back into austerity measures. Things are bleak out there, but if the state does crumble we owe it to each other to reject corporate dominion as an alternative.

Poisoning The Well

The one thing LLMs have managed to do with worrying efficiency is to completely conquer The Discourse. LLMs are like that one uncle with the worrying political afflictions who turns up at every family gathering. They’re incredible at turning up in conversations and making you have opinions about them. It’s become almost impossible to have a normal conversation that doesn’t mention the eldritch horror you can outsource your thinking to. I’m hoping that once the bubble pops we can all, collectively, stop fucking discussing this shit. Let’s hope it fades into obscurity like crypto discourse.

I’m hoping, no matter what happens, we are collectively freed from talking about it. In my mind, what this technology represents, more than anything else, is one of the biggest knowledge transfers in human history. Access to good information is increasingly complicated by LLMs, and those creating it have less incentive to keep going because they are competing with slop. It then harbours all of this information behind an inevitable paywall.

As a programmer, there is a great pathos wrapped up in watching a discipline I taught myself using open-source software and free learning resources be dominated by this. The very openness of the industry created a vast dataset that made it an uncannily apt target for automation. I can’t help but feel that large corporations took note of a group of powerful workers and saw a unique opportunity to steal their labour and give them a front-row seat at the slop factory.

As an amateur philosopher, I don’t think we’re psychologically prepared for the technology. The human mind has evolved to anthropomorphise and empathise. LLMs are a brain hack – by talking ‘like people’ the very act of communicating with them forces us to treat them like people. There is no way of doing so without being manipulated on some level. Relying on one to rationalise your own thoughts is to throw your cognition into the hands of an averaging machine. You will always be trended toward a median position. It may not feel like it but innovation dies here. And I say this as someone so exhausted by the discourse that I’ve been sitting on this article for coming on two years. Hell, I even used ChatGPT as an editorial consultant because I was so burned out – and had to fight the entire way to write the article I wanted to publish, rather than another smug, centrist take. I have no doubt that a real, human editor would have made this piece substantially better.

As an artist and writer, the technology feels like a loophole for fair use. It is an ‘ethical’ plagiarism machine that allows you to claim innocence because the input has been ground down to such a fine powder as to be unidentifiable. Fair use and copyright haven’t been designed for this. Not that I’m that much of a fan of copyright – it always asymmetrically supports the person with the most money. But this technology does not create novel ideas – not in my opinion – because I believe that requires consciousness and cognition. There are no new ideas, but that doesn’t mean all ideas are just re-assemblages of old ones.

If my next article is another critique of LLMs I think I’m going to scream.

Or maybe I can have a good chuckle about the “Magnificent 7” when they end up choking on it. I doubt that anyone’s going to burn to the ground but maybe there will be a black eye or two to go around.

In any case, one critical part of circumventing the deleterious effects these text extruders excrete is the death of the hype cycle. Hopefully there will be some breathing room for building solutions and fewer debates around the eye-watering energy costs of all these data centres dedicated to running black-box, general-purpose problem-solving machines.

It is easier than ever before to produce visual media, but what does that mean?

Since my previous piece two years ago, artificially generated material from LLMs (Large Language Models – the core of all modern generative AI) has only become more visible. To the point that it’s difficult to avoid an advert shilling for the newest ways to turn armies of expensive graphics cards into money. In that light I believe it’s time to take a second look at the relationship between AI and creativity.

As an artist, or someone who requires art, what do these services mean for you? It is undeniable that these services can provide a quick fix if you need ‘content‘ for a project you are working on; Almost like magic you can have something polished within mere seconds. If you need to impress and have few resources to hand then it can be incredibly tempting to reach for an AI tool.

However I would argue that in most cases these generative outputs are likely to hinder your project, especially if not used very carefully. The presence of this material will, ironically, only make your project more forgettable off putting and bland. You may also find in many cases that automating the creative process runs counter to the notion of creating in the first place. Finally, the hidden costs of AI might be prohibitive to you – depending on your priorities.

For the sake of focus I’m going to lead with the visual arts, but these arguments are still generally applicable to the other mediums.

Can LLMs make art?

Quoting a previous version of myself:

As with most areas of philosophy the definition of creativity is hotly contested. Though there is an emerging consensus that for something to be creative it needs to satisfy two conditions. Firstly it must be ‘novel’ – meaning a new combination of elements, original in some form. Secondly it must be ‘valuable’, though value in this context could be replaced with ‘exemplary’ or ‘notable’. It must have something to make that originality of wider interest – though this has been contested. Whilst there have been studies on AI and creativity, the limiting factor in current research is in-domain knowledge (to allow researchers to assess creativity within a medium), and valuation of results (the ability to judge the creative worth of the output). The complexity of the task of evaluation cannot be stressed enough – and alongside definitions of creativity, these tend to rely on factors like emotional response, motivations, lived experiences, shifting tastes, values and more. This is a running theme for researchers of creativity, alongside aspects of self expression, expression of of ideas, and a blending of them between the conscious and unconscious mind.

What I’m getting at is that theory of creativity tends to be connected to theory of mind and theory of consciousness. I propose that creativity has a direct causal relationship with consciousness. It is my personal belief that to be able to create in the way we understand true creation, one has to have a conscious mind.

This is why if you should choose to use AI services to create your own work it won’t be bringing any new ideas to the party. The lack of ability to generate ideas is why the only ‘value’ AI can bring to a work is brought by you and the context you bring to it.

Having said that, I do believe it is possible to derive value from AI generated work. As LLMs are trained on data siphoned from a biased society, the results are indelibly coloured with the strata that already exist in society. LLMs are a great resource for highlighting bigotry and various assumed truths – especially as western culture has a tendency to dominate the online sphere and is disproportionately represented in AI output. In this sense it can be used as a mirror, to look at ourselves. Having said that it is incredibly unlikely an AI image on its own will have enough artistic weight to be considered to have ‘true’ value.

Originality, and lack thereof

Something you’ve likely already noticed is that it is exceedingly easy to make painfully mediocre material using LLM’s, whether that’s midjourney, DALL-E, Stable Diffusion, etc. Facebook (for those who still use it) has become a ghost town dominated by boomer-bait ‘fake’ memes pumped out by pages dedicated to disseminating artificial images, and other social media sites aren’t far behind. You’ll notice that this AI slop tends to have a distinct look. To pick a random example:

Quite often the output has a hyper-glossy lighting treatment and has a tendency to look like an advert. The easiest way to understand why is to think of AI art as the result of taking ridiculously huge quantities of text and images, chucking them in a blender, and reconstituting that into output like the above. Fundamentally, an AI algorithm is unable to create or innovate – it is a very well designed mimic. It takes a request and pieces together a vast ocean of visual works to create a kind of hyper detailed exquisite corpse – a mish mash of concepts squashed together to make an impressive approximation of ‘content’.

Many argue that there is no difference between an AI algorithm using references to create art and a person using references to make art – but in my opinion this is a dishonest comparison. A human takes inference from reference material to create work that exists in dialogue with the reference. Through context and concept the reference material is used as a point of inspiration to make an entirely new work. In most cases, all of the actual mark making of the final piece is derived from the artist’s own hand. In contrast, the algorithm simply recycles the data of the original work into a new configuration – there is no interpretation because an LLM only has relational connections made during training, and no ability to create anything but a predictive guess at what the desired outcome of a text prompt should be.

This lack of context, awareness, ability to create a dialogue, imagination in short, is what makes AI content simultaneously so arresting and so forgettable. It recycles what has been popular before to predict what would be popular now.

Notice how Suhail focusses on details but there is no context or meaning here. Sure, there is an obligation for some art to be ‘beautiful’. But for art to truly speak to someone it usually needs to be imbued with some kind of meaning. And I think this is where AI highlights something important.

The Medium is the Message

The advent of AI art is not the first time a new medium has impacted the creative community. The advent of the printing press re-contextualised calligraphy – no longer pure communications technology but an expressive medium. The camera re-contextualised painting; taking it from a medium for stuffy portraiture to, once again, a tool for self expression. De-prioritising realism. How does AI art re-contextualise art? Or any other application it is turned to?

In a similar way I think it re-contextualises the very act of creation in the first place. It refocuses the creator’s act from ‘how’ to ‘why’. If you simply require something as opposed to nothing, and the actual material itself doesn’t matter then sure, it may fit your purposes. It simply allows you to create more – as long as the ‘more’ looks an awful lot like something you’ve already seen. It will always trend toward the average – the most common avatar of the concept you’ve requested, the archetype. It is an engine for derivative, ‘good enough’ work. This means that, in the creative sphere, the primary distinguishing factor – especially as the quality of AI outputs improve, will be intent and imagination. Why are you making this? What are you trying to achieve or convey? What are you trying to say? Is detail required to say it?

Heyheymomo is a creator of high quality eyebleach – aggressively wholesome webcomics. They are simple both in content and execution. They effectively convey incredibly human situations. In short they are infused with a sense of humanity that can only be created from a perspective of understanding humanity. It communicates on an instinctive level. In Spanish term word “Duende” is used to denote art that has life – a kind of soul imbued to it. Art with duende provokes an emotional response in the audience. Art that is alive. By comparison generative ‘art’ is is inherently dead, zombified even.

So can I use AI to help me make art?

Until an algorithm understands the qualia (i.e. the innate nature) of existence, it will not be able to impart true imagination to work – and in that sense never truly be able to ‘create’ anything. If the aim of what you are making is to say something new or specific to you, using AI will always be a guiding force steering you toward mediocrity. Of course, you can always use AI material as a starting off point, but still know that that starting point will always come from an average, rather than extraordinary place.

This might be acceptable for your purposes, though it will be difficult to use the master’s tools to demolish the master’s house. And considering what LLMs make is essentially a vertical slice of peak neoliberal culture, any direction you’re likely to want to go in is probably away from the centre. Unless you like beige, of course. I recently had to condense a piece of writing down from 5k+ to under 750 words and out of curiosity passed it through a chat bot. So much of the essential meaning of the piece was lost, and so individualised did it portray the problem, that in the end I rejected it entirely in favour of redrafting the piece myself. The scanner sees darkly, and if you choose to take on this vision, so too will it alter your perception.

So yes, if it’s a useful tool for you, then feel free to use it. But remember a real reference will always be more useful than something an LLM sticky taped together – and will give you greater cultural context because you will know where it came from.

The Cost of Entry

Aside from reasons around art theory and nebulous notions of creativity, there are other reasons you may decide against using AI to automate your creative process. Or parts thereof. The most widely covered aspect being the debate around intellectual property and theft. You may have certain feelings around whether ‘Intellectual Property’ should even exist in the first place – in a perfect world I certainly wouldn’t argue in its favor. However in a world of IP hoarding mega monopolies and asymmetric enforcement of those rights, copyright law may be the only line of defence against the wholesale stealing of innumerable artists’ work. As mentioned before – these services analyse petabytes of data, largely taken wholesale from the internet without obtaining consent from their creators. In fact, because of the way AI works, any output it generates is ultimately a kind of plagiarism.

Aside from this, the environmental cost of generative AI is staggering. Creating a new version of a model requires the equivalent power of a small nation to train – around 11twh and expected to rise significantly before 2026. On top of which, the power required to run these data centres adds even more power requirement, and many of the datacentres running the hardware use evaporative cooling – placing huge demands for water on the areas surrounding them. Some estimates suggest as little as 5 queries processed by GPT-3 can consume half a litre of water, and even suggest its’ successor (GPT-4) can consume the same amount for generating as few as 100 words of response

There’s also the human cost of the AI ghost workers who are employed en masse ensure the proper functioning of these service in what are poorly compensated effectively digital sweatshops. AI output can be unreliable and reproduce darker elements in the training data. People are employed en masse to act as a human crumple zone to ensure the output is sanitised – leaving these precariously employed people responsible if something goes wrong.

What Now?

I hope I’ve managed to convey the complex relation the AI economy has with the arts. If you still so choose to employ the outputs of these systems then hopefully you do so with full awareness of context.

I would like to leave you with one final thought though.

We live in an attention economy. Part of the reason the lure of AI is so strong is because it helps wrung out creatives produce more in a shorter time span – which is a demand made by our social media driven culture. If you don’t create often and in a certain way you lose visibility because you aren’t creating what the algorithm is looking for. As suggested in an article by Jason Koebler, AI is being used to brute force social media algorithms. To chase trends and spike visibility with whatever is popular at this exact point in time. And that does have power – it can be, and is, exploited.

There is a careful balance to be struck between achieving a goal and hitting a target. In the modern era the target increasingly seems to simply be ‘more’. But creating without intention, without taking pause to reflect on who you are, what you’re doing, and why, it can become empty. Especially if you stop enjoying it. Take it from someone who’s been there (and still is, to some extent). Don’t turn your passion into a job. Don’t kill your interest for the sake of the grind. Don’t destroy yourself in pursuit of an unattainable goal. And above all think. Make your creations out of passion and enjoyment. Create because if you don’t you’ll go mad. Create from the heart – however you do that, employing AI tools or not.

But beware.

He who fights with monsters might take care lest he thereby become a monster. And if you gaze for long into an abyss, the abyss gazes also into you.

Don’t lose yourself along the way.

My previous piece on AI can be found on my website https://gaffen.co.uk at https://gaffen.co.uk/posts/ai-ip-theft-and-the-death-of-creativity/. It delves further into ideas of creativity and machine consciousness and has a heavier focus on philosophy.

Content warning: this article includes description of an emotional breakdown and bodily self harm. It also contains mild discussion of the ending of the game Wanderstop.

Boro sat with me and told me I could take as long as I liked, and I could leave when I was ready. As we gazed across the grassy plain a gentle, melancholy piano piece lilted through the valley.

A feeling bubbled up within me and, not for the first time over my 15 hours with the game, I began to weep. But this time, realising there are no easy routes to healing from pain I began to gasp and raggedly sob. I had conclusively been invited not to learn a big lesson, not to fix myself a specific way, but just to acknowledge that healing takes time. That there are no quick fixes. That there is just the pain, how you carry it, and how you continue forward whether in its presence or not. After the credits ended I was still crying. I just sat and let the soothing ambience of the start menu play as I reflected on the struggles of the past decade and a half.

Wanderstop is a game about a fighter called Alta who has reached the peak of her career and suddenly starts losing fights. She keeps losing, and reaches the point where she can’t even lift her sword – something she sees as inseparable from her identity. Not being able to lift it is profoundly and deeply traumatic. Stuck in a wood and unable to do the one thing that makes her feel like she has purpose she encounters Wanderstop – a tea shop run by a simple, quiet man called Boro. The game is about listening to characters, making them tea, tending to the grounds and healing from trauma.

It’s painful and beautiful and one of the only pieces of media that has ever got me to actually openly break out into grief stricken tears. I think it’s what I needed right now.

15 years ago I started my first proper desk job. I was fresh out of art school and had told myself I’d never do this. That I’d be an artist – creative, free, unbound by the daily grind. That to sit in front of a desk every day slaving away on work I didn’t care about would be a kind of death. At the same time I started the band Hearse Pileup, and as I started to fully conform to the frequently 9 to 9 working hours of a creative agency that experiences the dreaded “crunch” I wrote the lyrics for the song “Grindstone”:

“I got my nose to the grindstone every day
And I’m grinding so hard I grind myself away
I get blood on the floor wherever I go
And if I grind much more I’ll be grinding bone”

Little did I know at the time exactly how true the words I’d written would be. I was severely underpaid for my labour. I was left with little disposable income and forced to move back in with my family; with my wife and I sharing my childhood bedroom. We both worked for the same company. Everyone told us that we should find alternative work but we were repeatedly convinced that the was the only realistic option for us. Stuck in a vicious cycle of despair, scraping by and spending what little we had left over to stay sane in an environment that was a pressure cooker of stress.

The entire time I diligently worked on the band. I would finish work, come home, and work until the early hours of the morning on branding, our website, promotion, networking, contacting gig bookers and venues. In addition of course to rehearsing and writing music. Every moment I could spare I would be thinking about it and what I could do to escape my work life. I played shows, which made me feel alive. I stopped going to almost any other shows but the ones I was playing because I couldn’t afford to go out regularly. We recorded music, we got involved in the occupation of the 12 Bar on Denmark Street. We rarely got paid more for playing than we spent on getting to the show and a handful of drinks whilst we were there.

And it never felt like enough.

“I have devoted every day of my life to this, every single day for years and years! You have no idea of the sacrifices I’ve made, the pain I’ve endured, you can’t begin to imagine it! Do you understand that fighting is the one single thing I’ve spent my life on? I CAN’T give up!”

“I am sorry Miss, I was not meaning to touch on such a nerve.”

“It’s not a nerve!! It’s me! You’re talking about ME!”

My work life didn’t improve much, and my frustration around lack of sufficient progression in my music career became an increasingly large weight. I can never know exactly how difficult it was for my wife to watch me go through this journey. But I do know that almost any time she wanted to have some personal time with me, going out on a walk, going shopping, going on a date, it started to feel like I had to choose between my relationship or my band. And I think I was responsible for that – I had shouldered too much burden and was allowing everything around me to atrophy in its wake. And worst of all I projected that outward and made her feel like she was making me choose. This is one of my deepest regrets. To this day, frustrations around my personal progression are something I have to keep a careful eye on. I don’t think it’s too much to expect to have a life in addition to a hobby. Or a career.

This culminated during the 2020 lock down in conjunction with some of the most heinous work hours I’ve ever experienced. At this point we were taking turns to lie on a sofa in our room in a silently tearful wreck whilst the other explained to our boss that the other was unable to work. We decided to go out for a peaceful walk around our neighborhood and an interaction with a random stranger caused an argument. It was my turn to be non-functional. I desperately persuaded my partner not to start a public argument as my constitution couldn’t take it right then. But what with her preexisting frustrations around people taking advantage of us that anger seethed under the surface and I couldn’t take it.

We got home.

We started arguing.

We started shouting.

I couldn’t take it any more. I’d taken too much pressure. I was too anxious, too burned out, too defeated. I couldn’t reason with the person I cared about most and I was hurting so deeply inside that I could scream… And I did.

I! JUST! NEED! TO! RELAX!

Each word screamed at the top of my voice.

I have a history of self harm, and every time I have done it it was to alleviate suppressed emotion. To relieve my emotional pain through physical pain. I swung my arm wildly. It collided with the side of my head. I did it again. I kept screaming.

I! JUST! NEED! TO! RELAX!

I kept going. I don’t remember exactly how my partner looked at this time but she must have been shocked. I fell to the floor. Words weren’t enough any more. I made a sound I never thought myself capable of, something I’ve not experienced before or since. A wordless, primal scream. I did it once, twice… and it began to dissolve into a wailing moan. I cried, my wife took hold of my hands. I stood up, still in tears, she held me. The side of my face had swollen up. She blamed herself – she said she’d done this to me.

But she hadn’t. She had been nothing but supportive and an extra source of strength for me the entire time. She wasn’t responsible for the conditions we’d been living under. Neither of us deserved that blame. Sometimes your actions are just the natural consequence of extreme conditions and the people who created them. I developed a moderately serious looking black eye. I wore my long fringe across my face to hide it from my parents. Like a domestic abuse victim. I now realise I was a kind of abuse victim, we both were. But it didn’t come from my home life.

My sister had been in the house, upstairs – I can only imagine how it must have sounded. I can’t imagine it would have been easy to hear.

I handed in my notice very soon after. I wore sunglasses at the time to hide my face.

Within a month my voice disappeared completely. The NHS told me there was nothing physically wrong with my throat apart from some mild acid reflux. It gradually returned but I couldn’t sing. And when I did start to sing again with the band I was locked into the growl of a metal singer – unable to maintain a clean tone. Something I needed if we were to finish off the album. We lost our 4th drummer. I stopped playing guitar regularly. I started to stop making creative work in general. I was drowning and I couldn’t see a way back to the person I was. We moved out to our own flat, we started to be able to afford a life together. Our material conditions improved but my wellspring of inspiration had mostly vanished. All the plans for drawing, writing, creating music seemed to turn to ash. And the things I did manage to do over that time felt like climbing a sheer rock face – taking huge amounts of effort to even make the smallest amount of progress.

I was burned out.

When Alta couldn’t lift her sword I immediately felt an intense pang of emotion. I knew that feeling. I began to tear up. I thought about my guitar. But all I could do was feel numb.

I feel that Wanderstop is a difficult game to talk about in the same way that Everything Everywhere All At Once is difficult to talk about. It’s divisive because it uses a type of double-frame narrative that relies somewhat on lived experience to be fully appreciated. Usually double-frame is used to describe a smaller story happening within a greater arc. In Everything Everywhere the greater narrative is about saving, well, everything. However the inner frame is where the emotional core of the movie lies. In Evelyn’s battle with the life she’s built and expectations of her father, and Joy’s desperate need for true acceptance of her and her sexuality from her mother. I am neither queer or the child of first generation immigrants, but have enough empathy and second hand experience from those close to me to pick up on the heartfelt earnestness of the way this inner, more meaningful narrative is told.

Similarly, Wanderstop is a game that on The surface presents itself as a cosy tea shop management simulator. But beneath that is an extended therapy session about burnout, emotional trauma and letting go. One that I think you need some understanding of the the real feelings of recovering from trauma to even be receptive to.

I don’t think the divisiveness is the fault of the game’s trailer, or its aesthetic design, but because “cozy” has become such an industry. There is a high demand for relaxing games you can unplug from the world to and just… be somewhere else for a while. And you could call this a kind of burnout too – many reviews of Wanderstop mentioned ‘feeling burned out’ by playing it. But the sort of burnout the game is addressing isn’t feeling exhausted after a tiring week – which is still valid. It’s the level of burnout you feel after pushing past that boundary. The kind of burnout where you put every ounce of your being into something only for it not only to not get better but actually get worse. When the thing you loved becomes a source of pain.

And acknowledging that is uncomfortable – Davey Wreden (the games director) knows this, I’m certain. The slow, ambling, meditative pace, the lack of direction, it all kind of forces you to sit with yourself. Wanderstop isn’t a cozy game – it’s a digital therapy session. And you have to be ready to deal with some serious interior emotions as the cost of entry (if, indeed, you have those emotions to process).

As an aside, another element the game deals with is the lack of agency in keeping things you might feel attached to. Accepting that, no matter what you do, sometimes people just disappear from your life and there is nothing you can do about it. A couple of years ago Hearse Pileup’s first drummer, the guy who named the band, passed away. We’d dropped out of contact. I wish I could have talked to him one more time but it didn’t happen.

Someone in my family I had put massive amounts of effort into helping through a difficult situation, similarly, removed me from their life.

Sometimes, no matter how you try, you can’t help someone and you can’t get the satisfying end to the story you see in movies. Things just end.

So, Wanderstop, to me, was an incredible experience that forced me to truly, clearly, look at myself. I don’t know if I’d even call it a ‘game’ in the traditional sense. And you may well not get out of it what I did, your mileage may vary. But there is something very beautiful and clearly observed at the centre of this game. I can’t understand exactly what Davey went through in the production but I’m certain he must have experienced his own fair share for burnout related trauma to end up where he is.

All I can say, really, is thank you to Davey for helping me through mine.

Since starting this article I’ve started picking up my guitar again.

What if I told you that, like almost everyone else you know, you are likely a slave. You are trapped in a prison you cannot smell, taste or touch. A prison for your mind.

You’re reading this because you know something. What you know, you can’t explain – but you can feel it. You’ve felt it for years – that there’s something wrong with the world. You don’t know what it is, but it’s there. Like a splinter in your mind – driving you mad.

But unlike the Wachowski sisters’ Matrix, you weren’t born into this. Instead you silently opted into the largest sociology experiment ever created. You didn’t know that’s what you were doing – but it’s what you did. The prison you reside in is present on your smartphone, on your smart TV, your smart watch, smart thermostat, smart gym equipment, smart mattress. If it’s smart, likelihood is it’s all part of a gigantic cage made to your exact specifications.

Your bank, your health provider, your government. All these things conspire to trap you. And it sounds insane, like a conspiracy, but it’s terrifyingly real. You almost certainly already know information about you is used by services attached to all these things. Quite a lot of them wouldn’t be able to work without knowing something about you. What you maybe don’t understand is how they work together to trap you.

Like any manipulator, the process of entrapment is steady and calculated. I’ll be outlining three major forms this takes: influencing your movement and actions, disrupting your relationships and reshaping your identity, and finally, training you to actively resist breaking out of the system.

So you have a choice. You can stop now, go back to your day, lead a normal life.

Or keep reading, and we’ll see how far the rabbit hole goes.

Nearly 99% of test subjects accepted the program as long as they were given a choice

A recurring theme here will be the overlap of using and being used; a double movement of influence. A lot of the ways online networks affect your actions seem quite benign. You share your location with google for helpful directions. Instagram shows you a popular viral popup store and you decide to go. A location-based mobile game tells you of a high-value item nearby. You haven’t been forced to do anything, you were just interested in something and were given some gentle suggestions.

However, if you used Google’s “live view”, your camera feed is used the update their street view data. Meta creates a profile of you to know the exact time and place to show you an advert, or a post that is also an advert, to influence your actions off-platform. Niantic, the company that makes Pokémon Go, allows businesses to purchase the placement of significant game items near them to guarantee foot traffic.

But I’m not telling you anything new here am I? This is just business, nothing comes for free. If you don’t pay for the product, you are the product. This is the era of the platform economy.

The Platform

The word “Platform” is an interesting one – both a constant presence in our daily lives and nebulous. A noun for a non-space, a void awaiting your presence. It’s worth digging into, and its modern usage was born shortly after the internet.

When the web really started taking off in the 90s, it was very normal for users to build their own websites. It was DIY. But not everyone had the required resources to host them. When it came to making financial transactions almost nobody did. This is why eBay became so successful. It provided a service for other people to use, and took a small cut of the profit. Thus the modern meaning for platform was invented – an online space that exists as a for-profit entity that provides the illusion of a public service. A digital mall, where you’re free to do what you want as long as you follow the rules and can afford the cost of entry.

Now almost the entirety of our online lives are spent mediated by these platforms. The social ones are the most overtly present in people’s lives today; finely honed attention-monopolising machines. They use gambling mechanics to wire your brain into a Pavlovian feedback loop of addiction; trying to keep you logged in and paying attention to them as long as possible. This is in the platform’s interest because it makes them money. And in doing so this two-way relationship begins between yourself and what you consume. You are what you eat.

Machine learning, fundamentally, is very good at a specific kind of problem solving – categorisation. It doesn’t matter what political affiliation you have, whether your beliefs are strong or weak, or how “objective” you think you are. The only thing that matters is how an algorithm can fit you into a neat little niche and get you to stay there. It’s not that exposure fundamentally changes you at any level… It just sands the corners off your personality. Gently encourages you to fit into your perfect niche.

The insidiousness of this is, no matter what position you take, you can be neatly pigeonholed somewhere for people just like you. Strongly politically aware or completely ambivalent. Right or left, authoritarian or anarchist, cats or dogs. And if there isn’t a category for you, you’ll be slowly nudged into the one that is your closest match.

This is done at a variety of levels. The most obvious is targeted ads – targeting you by location and interest, calculated to reach you at your weakest and most susceptible moments. There’s also your algorithmic feed, showing you whatever it can to hook you in. Whether that be outrage, an amusing dance, a tempting argument, a piece of clickbait. It produces an intoxicating cocktail of dopamine and cortisol designed to make you feel, as much as possible, that your online life is your life. It doesn’t end there either, it seeps out into legacy media and ‘real’ life. News outlets need to rely increasingly on clickbait tactics themselves to succeed. And the ‘culture war’ seems to worsen in proportion with it. Politicians are now way more scared of bad PR than they are of doing the right thing.

And all of this funnels you, ever so gently, along algorithmic fault lines. Surrounded by caricatures of your own opinions, positions, likes, dislikes. And the more you’re exposed to it, the more you are encouraged to express yourself similarly.

Sometimes the fault lines start crossing the family dinner table. And the more isolated you feel, the more you turn to a place of familiarity. A place where you know you’ll have a sympathetic ear.

This isn’t an echo chamber, it’s much more insidious than that. You’re in a cell of an elegantly constructed panopticon. One that feels like the life you chose, It’s just all the edges have been filed off to make you easier to predict. To better fit you into the standard model for your category. Not even its architects clearly see the floor plan. And as long as you act the way they want you to (locked into the platform, pleasing their adspace and security customers), they don’t care to. In fact, the way these algorithms work, it may never be possible to see the invisible pigeonholes people are filed into.

Beyond the silent process of categorising and isolation is an even more insidious force. The intricate mechanics of the digital economy also trains you to flatly deny that the cage even exists. Even when your face is pressed up against the bars.

I See Six

In George Orwell’s 1984 there is a scene in which Winston, the protagonist of the story, is mercilessly tortured. His inquisitor, O’Brien, is ensuring he has total ideological control over Winston. He holds up four fingers and asks Winston how many he sees. The correct answer is however many “The Party” (in essence the state) tells him to see. Eventually, the pain causes his senses to present him with the illusion of innumerable fingers – and he can only say “in all honesty I don’t know”. When he gives this, the desired answer, he is administered a painkiller that floods his body. He feels instant gratification and love toward his torturer.

It is facetious to claim that we are exposed to that level of direct manipulation, but this scene illustrates a pattern of domination. It’s reminiscent of the techniques employed in coercive relationships and cult indoctrination. A very common trait of this indoctrination is exploiting and/or fostering a disorganised attachment style. I won’t go into excessive detail but the basic idea is that you isolate your target and make yourself both the source of traumatic events and comfort. Doing this takes away a sense of stability. Without a clear way to live in a safe and stable environment, the victim has nowhere else to go. And this partners well with a community that reinforces or normalises the victim’s relationship with you. So the victim submits, resisting is too painful and life is so much easier doing so.

It only take a small extra logical leap to see how this relationship pans out in the online sphere. The relationship between usage of social media and poor mental health outcomes are very widely documented indeed. Whilst it would be unscientific to claim that user relationships work in exactly the same way, a similar pattern can be observed. The chaotic mix of irritants and ‘good’ content that sucks you in, it bears a similar resemblance.

And though you still talk to your friends and family, the platform becomes the mediator of all these interactions. These websites are perfectly positioned to inject whatever they like into your feed. Because you have to be there anyway, you are given no choice but to look at it. Even if you’re not isolated directly from your social circle by it’s online counterpart, you are forced into your assigned category by being present on the platform.

So if these services are so bad for us, why don’t we stop using them?

Opting out is hard. Really hard. The presence of your social circle on these services forces you to be there too, and it extends way beyond what would usually be seen as social networks.

You have facebook, or twitter, or snapchat, or tiktok because your friends do, and it’s so much more convenient to keep up to date with them by looking at what they post rather than talk to them. And if you don’t, you’ll feel left out when everyone else is in on the most recent development and you aren’t. You have amazon because it’s so much more convenient than buying from individual stores online. You have netflix or disney+ or appletv because it’s so much more difficult to watch what you want to without them. You have the same online banking app as your friends because it makes transferring money so much easier. You have spotify because why pay for music when you have access to almost anything you’d ever want for free, or as close to free as to be a distinction without a difference. You have whatsapp because, well, how else will you talk to your friends?

But, there are ways around this. You can buy physical copies of media, you can shop local, you can close your social media profile, you can only use SMS or Signal (matrix/briar if you’re really cool), you can use cash everywhere. However if you try to do this you will immediately encounter a lot of friction. If you have ever tried to ask your friends to switch over to Signal you will be acutely aware of this. If you don’t have spotify, how will your friends request music at your next house party? If you don’t have every single streaming service on the market, how will you keep up to date with the most recently trending show? People you know will increasingly react with annoyance if you aren’t using the same services as other ‘normal’ people. And for this reason you are quietly, gently, but forcibly pushed back into the circle of influence of online platforms.

And finally, once you start using one of these services you are entrapped. Your data, your photos, your interactions, your playlists. Everything that you might value that is created on these platforms becomes stuck there. Yes, you can request your data as an export – but when you get it what will you do with it? How will you view it? And inevitably it won’t be complete as half of ‘your data’ might be photos of you on other peoples’ profiles. And if I want to see your family photos on these services, most of the time I will need a profile myself if I’m to view it with any kind of level of ease.

All these barriers artificially increase the cost of opting out of the system of manipulation.

A computer-generated dream world built to keep us under control

Hopefully I’ve started to allow you to see the bars of this digital prison, even if it goes to great lengths to convince you it doesn’t exist. You may still be unconvinced as to the intentionality of this – surely that’s just market capitalism? Services have to compete. And yes, this is how capitalism works – though ‘true’ market capitalism is apparently anti-monopoly… somehow. But in terms of intentionality, not every service connected to the internet necessarily even knows it’s part in the system. Platforms like facebook or google certainly do. But the independent web shop you bought from who uses an analytics service to understand how well they’re operating likely doesn’t.

The issue is the privacy policy you signed by using the site is more like an uncontract. A dubiously legal waiver that, in almost every case, assures you of your privacy whilst mentioning ‘trusted third parties’ they share your data with. (Dubiously legal as it it is legal, but the law has difficulty keeping up with tech.) This loophole is an infinite regress of data processors that in essence allows your anonymised usage data to be sold to the highest bidder. That data can be correlated with other bits of scooped up information and be deanonymised. Given access to every online activity, your geolocation, your movements, card transactions, viewing and listening habits, all the same data from your friends… It wouldn’t actually be that hard to identify you. You just need to be able to process vast oceans of data… Something that, say, a search engine provider would be uniquely talented at. facebook only needs to have 300 ‘likes’ from you to know you better than your spouse.

This is how the entire system of digital systems is able to trap you and mould your movements and the way you think. Little micro nudges, friction, inconvenience. But simultaneously being everything to you, all the time. Inserting an internet connection (a surveillance tool), between you and almost everything you care about. Then weaponising what would have been military grade identity profiling technology to push you to make as much money for them as possible.

It’s this implicit buy-in, the click-wrapped agreement to terms of use / privacy policy that has become so ubiquitous as to be invisible. It’s a flimsy excuse to create a binding legal agreement to legal documentation that averages out to about 250hrs reading time per user; granting them permission to use your information as they see fit. The users are presented an environment where they are made to feel like they have control – that it is their space to shape and build, but this isn’t the case. These aren’t digital town squares, but digital shopping malls. A for-profit company owns the platform you’re using, and you’re ultimately at it’s mercy. Even critical infrastructure feeds into this; the UK’s government website uses google analytics. The data created is shared with third parties that is prime for deanonymisation. Even your bank is likely selling your data.

This primes you for influence by an unknowable cabal of disparate forces, all driven to make the most of private information and extract as much value from you as possible.

The upshot is not a specific outcome, as mentioned – these companies only really care about their bottom line. It’s not intended that Meta aided the Myanmar genocide, but doing so generated the most profit. It’s not the goal of social media websites to promote the rise of fascism and white supremacy, but it makes money. Whilst it isn’t the only place the pattern is visible, it is specifically in the rise of the alt right and stochastic terror attacks that it is easiest to see. And in being so visible, it also, as implied earlier, most visibly mirrors the recruiting strategies of cults.

It builds a kind of self-reinforcing authoritarianism.

Stochastic Totalism

Stochastic (as in a result of random probability) terror is a kind of randomised violence where a group of people buy into and help spread a violent narrative. The message is spread multilaterally – at any level of media and at any level of intent. However the result is that the ambient levels of anger toward a specific group of people eventually bleed out into real life, often in deadly fashion. These groups will choose authority figures who may not even recognise themselves as such, but who are nonetheless held up as figureheads. It doesn’t matter if they’re right. It doesn’t matter if they’re even on the same side. It only matters that they pass the vibe check and embody a narrative. Ian Danskin has identified this behaviour as stochastic totalism – and I think it’s an incredibly important observation. But I don’t think that framing quite captures the entire picture.

We are all being pulled into our own version of this. Maybe not quite so violent. Maybe not quite so obviously vitriolic. But the centralising, categorising, oversimplificating nature of algorithms designed to maximise profit means we are all shunted into prefigured thought groups. We are all encouraged to find our own thought leaders and authority figures. And we’re given our own virtual neighbourhoods of people who agree likewise – ones we self-select because we are guided to do so.

And because these stochastically-generated affinity groups autonomously grow so big, they can be weaponised to brigade mainstream media. They become forces powerful enough to force politicians into capitulating and conciliatory roles. Government leaders become stewards for their preferred flavour of the status quo rather than having ideological visions and firm plans. It mirrors the capitalistic intent to siphon as much profit from as little material as possible. Never creating anything, just grinding existing assets down until there’s nothing left.

Depressing, isn’t it? But that’s the point.

Doomerism

Since the fall of the soviet union and the neoliberal decision that history is over, we’ve been living in an infinite now. Fukuyama claimed that representative democracy is in it’s final form. We did it, we solved government. And in the decades since 1991 we’ve seen exactly how stable and effective the ‘true form government is’.

This attitude, that we have reached a point where the tools with which we wield state power are perfect, lends to a focus on plugging holes instead of long term solutions. Market capitalism, likewise, is seen as a natural good. The only reason this would ever be the wrong is because of crony capitalism. It states that this is as good as it gets – a viewpoint that corporatism thrives on. Whether it’s true or not, the spectre of ‘if we only did capitalism good everything would be ok’ is an excuse that insists that humanity has finally worked it out, that all remaining problems are individualistic. By indoctrinating this mindset it allows capitalist interests to hide themselves as targets. It is in their interests that we don’t question the dominant neoliberal narrative.

Visions of a dystopian, corporatist future are partially popular because they don’t challenge us with change. Being resigned to our own inevitable demise means that we don’t have to worry about fixing it. We know there is no future, so we don’t have to worry about it.

We know that solving climate change is too big, that a few recycled or reusable plastic bags is a drop in the bucket. Yet we are told that it is all of our individual responsibilities to reduce our ‘carbon footprint’ (a term invented by the oil company BP). It’s too much responsibility to be held to – we know that, individually, we cannot do this. And we are kept overworked in order to reduce the amount of energy we have to even think about it. We’re all just trying to survive, day by day. And this scales upward – most corporations are just trying to cover their bottom lines and trying to ensure they have enough money to return to investors. Even the government will hold off on investing in solutions that will come to fruition in more than a few years for fear that they won’t be in office by the time the results of trying to repair or improve systems pay off.

This is relevant because with the advent of social media there is no way to escape the news cycle entirely – it gets pushed into users’ faces, often without warning and at unexpected times. News apps are built into phones to ensure you are shown the right thing at exactly the right time to bait you back into the doom cycle. Just enough. Online arguments exhaust people and push them away from politics, and this view of society encourages us to stay in our respective lanes.

Pursuit of profit at the expense of everything else has led to some of the most precarious working arrangements ever conceived, with the rise of the gig worker. And gig workers are now fully integrated into our society. Deliveroo, uber, better help, even a lot of AI work is actually done by precariously employed individuals with no contract and no job stability. The work of creating entertainment is also slowly being turned into gig work, with youtube and spotify putting pressures on musicians, documentary makers and video essayists to crank out material on a regular basis or face harsh penalties in their distribution. Even then the relationship to stochastic totalism means that if they fall afoul of the algorithm they may suddenly lose security anyway. The pressure to follow the curve of totalistic demands means that you have to appeal to the sect you’ve been assigned to – without even necessarily knowing who they are.

These pressures lead to a population that is burned out, exhausted, and disproportionately concerned with present conditions over long term plans. It’s survival. It’s a state of hopelessness that constrains our ability to collaborate – it is designed to disempower us.

At each level of scale it seems that the production demands of stochastic totalism (hand in hand with market capitalism) drives this bend to the middle that crushes creative thought and independent thinking. Sequels, remakes, spinoffs, the creative industry is being crushed under the weight of intellectual property franchises. Market dominance by monolithic rights holders, guaranteed sales returns that starve out drives to pursue original ideas. Major music labels now only really invest in guaranteed investments – not in developing potential.

It’s a pattern that bears concerning similarity to the behaviour of ‘stochastic leaders’, people like Jordan Peterson, Ben Shapiro, Russel Brand, Alex Jones. Donald Trump. Self selecting feedback loops that are based on nothing but profit maximisation and consolidation of power.

Capitalism is a death cult, and I personally don’t believe any level of regulation can ‘fix’ this phenomenon. To the point that it extols its own narrative; it is easier to imagine the end of the world than the death of capitalism. The entire mechanism is designed to make you feel disempowered and helpless. The ‘doomer’ mindset is its primary tool of self defence.

You can’t correct for hate

The problem with taking a reformist approach to issues like these is that you inevitably end up in a position equivalent to the paradox of tolerance. Free market capitalism and resisting monopolies run counter to one another, and monopolies are a natural result of trying to achieve market dominance. When selfishness is rewarded so handsomely by the design of a system, I can’t help but see this as poisoning the well in the same way that allowing the truly intolerant equal grounds of the tolerant does. The double movement theory places individualism and socialism on two sides of a scale. But when one of those forces (intense freemarket capitalism) is able to crush the majority of the population into poverty, whist the other is looking for equity, it is clear that it is incompatible with continued existence.

A similar approach is being taken to automated moderation systems, content selection algorithms, AI Chatbots, etc. A common view is that these programs are in their infancy and can be corrected and improved incrementally – a kind of AI reformism. Essentially the systems that make up the backbone of the digital (‘platform’) economy. Bad, negative, or unhelpful patterns can be weeded out. In some cases this can be true – if the problem is very simple, there is a good chance that, fairly reliably, you can use machine learning to solve a problem. Such things as efficiency calculations for walking systems, shortest paths, and other evolutionary algorithms have a fixed set of inputs and a output that can be easily assessed for success. Facial/gestural recognition has also been a place this has been used fairly widely, and even though this is still a fairly simple task in comparison to others made today these systems are routinely less capable of correctly identifying dark skinned faces than light skinned ones. In automated heath care assessment systems powered by AI, similarly, those from a more diverse background are routinely marked as at higher health risk than those from a white background. The same can be said of the automated benefit fraud prevention tool used by the UK government, that cut the benefits of some of the UK’s most vulnerable people because they fit a pattern the AI model saw.

The problem is that even those building these systems can’t actually explain how they work – they are built by pumping loads of data into a training system. The AI system then makes inferences after being shown huge amounts of examples and asked to perform tasks based on them. But because the system doesn’t ‘see’ or ‘think’ like a person does, there is no telling exactly what it picks up from the data given. But the data they use to train these tools comes from a world that is saturated with things like structural racism, sexism, any kind of bias you could name. They aren’t just in the data – they are the data. As a result, data from an imperfect society will train an AI to do things imperfectly. But this becomes a problem when these systems are presented as science. Presenting these systems as ‘using science’ gives them a veneer of a ‘view from nowhere‘. If it’s science, then it must be logical, unbiased, cold and calculating. The screen of science is used to protect a system that does nothing more than amplify existing biases in society.

Just like with the double movement of capitalism, it seems as if the thinking is faulty. You can’t train neutrality out of bias. The systems being made to automate your work, your healthcare, your policing, your newsfeed, they are all predicated on nothing more than making money. And not only that but they are doing it by stratifying society into sectors suggested to it by the pre-existing stratification of our world. A real solution needs you to unplug. A real solution to these problems requires people to step out of ‘the matrix’, to have diverse conversations between peers and affinity groups. To break out of this system requires a voluntary step back from this kind of technology and towards one another.

Outside the Asylum

One image that keeps coming back to me when I discuss this stuff is one of the ‘Outside of the Asylum’ in Douglas Adams’ book So Long and Thanks for All the Fish. In it, a character called Wonko the Sane builds an inverted asylum. It houses the entire universe ‘inside’ the asylum and reserves a small square patch of garden ‘outside’. This is to get away from the madness of a world that feels like it needs detailed instructions on the use of toothpicks.

Explaining all that I have just explained to you here feels exactly like that. It’s absolutely crazy. It’s massive. It almost feels entirely inevitable.

But I can assure you that it’s not.

Unfortunately unplugging from this ‘matrix’ isn’t going to be as easy as taking a red pill. Ultimately it will be different for every person. And the way these systems work is, as mentioned, specifically designed to be hard to resist. Just like preventing climate change there is no one action we can do to stop it individually. (Although there are probably some individuals who could but absolutely won’t). Instead what it requires is understanding, consciousness, and consistent effort. If you can easily stop using a social network, do so. Or find one a little less rubbish. Try to minimise the amount of app mediated services that run your life. Start using alternative youtube frontends and alternative clients. Pay with cash or your bank card, rather than your phone. Use a privacyrespecting phone, divest from FAANG service providers. Physically meet up with other people in real spaces. Get involved in your communities. Dan McQuillan suggests People’s Councils – specifically focussed on AI but also generally worth embracing. Specialised niche interest groups of people affected by a specific issue. Learn how to run your own tech solutions or find people interested in helping you do so. The only reason these platforms have so much power is because of ease of use. If we could all accept a little more friction we might stand a chance of collectively unplugging.

And once we do that, we may find ourselves in a world where we once again feel more truly free than before.

A lot of research went into this article, but I’d like to specifically reference the work of Dan McQuillan, Ben Tarnoff, Shoshana Zuboff, and Ian Danskin as primary sources.