Ashraf Johaardien: The Rod of Asclepius, potholes, pain, the caduceus and other not-so-comic system errors.

I ask the receptionist if I need an appointment.

She says just walk in. I say great. I ask if they just beam the results straight to my doctor. She doesn’t quite laugh.

I follow the app to the address it gives me. Ten minutes later, I am driving in the wrong direction because the app directs me down every single pothole on the street. Each specific. Each avoidable. None of them avoided. I park somewhere. A guard waves me toward a building. A nurse squints at my paperwork. She tells me I am in the completely wrong building and need to go next door. Next door has a gate. I press the green button. The gate opens, I walk inside.

Nothing here has actually failed. The receptionist answered correctly. The app found real roads. The guard pointed at an actual building. The nurse read the actual form. Every part of this is working exactly as designed, and I am still slightly lost. The system is working. I am simply unable to understand how.

Inside, a cold steel chair. A nurse. An arm. A needle. Two glass tubes filling in the particular unhurried way blood fills a tube, patient about it, just leaving. This is where the errand stops being administrative. Something is now literally inside the system. And the system is now literally inside me, a needle finding a vein the way the app found a road.

The next morning, I wake up burning up with a high fever. Can’t get out of bed for two days. I’m not claiming the blood draw caused it, and I have no way of establishing that it did. What I can say is that the errand I completed successfully was followed by two days my body spent registering something the errand had no column for. The form had a box for my name, my date of birth, the reason for the visit. It had no box for what came after.

What does a system register, and what does it fail to register?

The app doesn’t know the road. It knows a model of the road, built from other people’s driving, and it hands you an instruction with total confidence, and only the potholes under your actual tyres tell you the model was incomplete. What happens once the model isn’t built from other people’s driving anymore, but from other models’ driving, one remove further from any road an actual tyre has touched.

Still in bed, I read about a 2024 Nature paper by Shumailov and colleagues, which turns out to be exactly that question, answered. Training a generative model repeatedly on other models’ output causes something specific to disappear, not everything, the tails, the rarer material, the edge cases, quietly pruned out generation after generation until a distribution that once had real range narrows into something thinner.

The useful part isn’t the doom in it. It’s the recursion: an output can become part of what the next system learns from, so material that started out human passes through, comes back out the other side, and some of what comes back gets folded in again as more material to learn from. Not a line. A loop. That’s the mechanism.

Operation precedes recognition. Last week, in a smaller, noisier version of the same thing, Mark Chen, OpenAI’s chief research officer, put a number on it, telling a reporter the company was eighty percent of the way to something they’re calling general intelligence. Greg Brockman wondered whether people will look back on this period as the moment it arrived. What Chen actually gave was a measurement against a boundary he and his own company get to define, not a description of anything the system does differently tomorrow than it did yesterday. Perhaps the difficulty was never that we don’t know what these systems will eventually do. Perhaps it’s that they are already doing it while we are still arguing about what to call what they are. I’d want that tested.

Separately, the IMF already estimates close to forty percent of global employment is exposed to AI, complementing some of it as readily as displacing the rest, the split between the two not yet settled. The recursion tells you how the loop works. The forty percent just tells you how much of the economy might eventually feel it, one way or the other.

Which raises an odd question about the hospital. Is a culture starting to resemble that morning’s infrastructure, competent at every local step, and still hard to navigate as a whole, because each part is now operating on material some earlier part already altered.

Blood leaves a body. Is processed. Interpreted. Becomes information somebody else acts on. It doesn’t come back, though. Not into that vein, not into that body, not ever. Whatever a culture makes, an essay, an image, a sentence, doesn’t carry that same guarantee. What happens once some of that material was never a body’s to begin with, generated rather than drawn, and it doesn’t just enter circulation once the way blood does. It re-enters, alongside everything a body actually made, indistinguishable soon enough from the inside of the loop.

We cannot diagnose what we refuse to recognise as part of the body. I don’t think that makes the system an organism. What it might mean is that culture is the kind of environment where an output becomes an input regardless of whether whatever produced it was human, and treating the system as something wholly separate from the culture producing and consuming it may be exactly what keeps the diagnosis from landing.

I got where I was supposed to go. The blood got where it was supposed to go. The system had done what it was designed to do. It was only later that my body told me there was something else to know.

Something the paperwork was never built to hold.

Ashraf Johaardien writes under the M\e. imprint. He is the Resident Writer at the Fak’ugesi African Digital & Innovation Festival.

Ashraf Johaardien
ashrafjohaardien@me.com
M\e.
http://www.ashrafjohaardien.substack.com

Disclaimer: Artslink.co.za encourages freedom of speech and the expression of diverse views. The views published do not necessarily represent the views of Artslink.co.za.