On Sep 3rd, the launch video for GPT-6 Astra was released, and along with it came 20-day worth of pent-up spite in my body. I must now write a blog about how sickening that launch video was in order to fully detoxicate and resume my otherwise bubbly personality. The droning preach of “if you don't have something nice to say, don't say anything at all” is not unfamiliar to me, but I also happen to know that “saying not nice things makes you feel a lot nicer.” So here comes my word diarrhea.
If you haven’t watched the video and are currently looking for ways to sour your day, you may check it out here at your own expense. I think the moment that really did it for me was at 0:42 mark when someone swaggers onto the set, then asks Astra to whip up a presentation for what the next season’s retail rainwear should look like.
The following words were fed into the machine…
Make sure that it feels really high-end and that it’s colorful. Make it fun.
Make it fun. Make it fun. Make! It! Fun!
Ok Astra. Now What The Actual Foundation model do you know about “fun”? By “fun,” do you pretty much mean "colorful”? Or are you gonna refer to my last chat where I asked for fun movies and picked Spider-verse? Or perhaps you will search the Internet for fun high-end brand campaigns like #TFWGucci?
To be fair, I shouldn’t insinuate that you don’t know what fun is. In fact I just spent a couple cents and asked you myself, to which you responded “what feels fun depends on people’s mood, preference, and company”. But of course, you didn’t ask that lady what kind of mood, preference or company she was in before jollying along with your terrific Google Slides skill.
And what do you know? The character stood bathing in your magnificence and said
“Oh, I love it.”
If I must guess what she loved, it’s probably your bold employment of the default Arial font and the playful contrast it brought to this clearly high-end rainwear brand. This is why we can’t have nice things anymore.
A life spent verifying outcomes
So what’s the big deal? Outcome achieved, mission accomplished, objective complete, what else could I possibly want? After all, the lady already said she loved the presentation! Fun has been computed and all the Naviers are Stoked!
But…(sigh) not everything is just about the outcome, and if you are not careful, then almost anything can be. I’m not even talking about how people derive meaning or joy from the process, I’m talking about how dangerous it is to go above your head. AI is peddling a dream where everyone can be an architect, a designer, a judiciary of taste, but you can’t truly appreciate the subtleties that give birth to greatness without having seen how it’s done.
You can’t speed run craft, taste or wisdom. There are sparks that only get revealed when you yourself have gone through the journey and its many side quests, which is also what Terence Tao and 25 other Field medalists wanted to illustrate in their declaration A Severe Misalignment of AI in Mathematics.
In recent months, the success of AI in solving major mathematical problems has made headlines even outside mathematical circles. But solving problems is only a tool and proxy for achieving the primary goal of conceptual understanding and insight. Forgetting this in the world of AI may turn the tool against the primary goal. Indeed, the mass production at faster and faster pace of “true/false” statements could destroy fertile ground instead of breathing life into new ideas.
It is incredibly easy to press on while not fully understanding the difference between using a calculator and becoming the calculator.
Do not outsource stochasticity
Btw, I fully acknowledge the irony of anthropomorphizing Astra by using the word “you” a lot, so I shall replace it with <you> going forward like a good human tokenizer.
My fellow technologists, don’t kick me out of SF yet, I’m not anti-AI at all. In fact, it is virtually impossible to do work without <you> anymore, but I try to be cautious about what <you> are doing to me and doing on behalf of me. When I say “me”, I primarily mean
My thoughts (part of the conscious mind)
My instincts (part of the unconscious mind)
Naturally, there are activities that do not require thoughts, like “draw a person in two seconds, one stroke, don’t stop.” I would literally go lights-off and watch my instincts glide across the unconscious blankness. Then of course, there are activities that do require thoughts, like “draw a person, but take your time.” Either way, both drawings are mine and mine only, every differential on the curve unique.
This reminded me of this scene in Ghost in the Shell — a human asked a cyborg why he was recruited when cyborgs seem to be perfectly capable of everything, and the cyborg said that it’s because an organization perish when they homogenize. It almost seems ironic that in a 1995 sci-fi anime we had the clarity that machines would want human’s heterogenous randomness and thirty years later we want to remove that as much as possible.
Our brain is this beautiful noisy soup jello and that noise is who we are. Outsourcing that stochasticity to a pseudorandom mathematical model is but heat death to our soul.
Btw yes I know inference hardware introduces some level of true randomness but the infra is literally designed to de-noise that very randomness so…
It’s not <you>, it’s me
When I do something all by myself, it feels continuous, like a fish swimming around that iceberg of mine, diving in and out of the cerebral soup. Yet when I work with <you>, it feels discontinuous, interruptive or almost pixelated. It goes something like this: I ask for something, <you> start doing something, I wait for a bit, <you> finish, then I react.
It feels like time traveling sometimes, and, after a while, I no longer experience time in Planck units but in chat turns. The promise of doing more with less, the insane amount of work automated by just doom-approving (and yes I’m a luddite, I don’t auto-approve everything), my monkey brain is being reward hacked to goopity-gloppy-gunk.
Astra, I must apologize, it’s not <you>, it’s me. Seduce me, reduce me, produce me.
What are the rules to advertise an addictive product anyways? I was referencing some old cigarette ads and how they all want you to associate a wonderful life with substance, and that got me thinking — maybe the launch video is so intentionally uninspiring so people won’t be fooled into thinking that AI is the answer to everything… Oh Astra, I’m just disappointed, out of a million marvelous things you could portray, you showed us how to automate away the meaning of fun.
Outro
The opening frame of the launch video referenced the demo of Put That There, an early gesture-language interface from MIT Media Lab. What stuck with me was, towards the end of the demo, the researcher told the system to put a large green circle on screen. For whatever reason, the system generated a red one instead. There was then this moment of silence, before the researcher dropped his head and sighed “Oh shit…Restart.” That moment of silent frustration is oddly what resonated with me the most.
<You> would never have shown this clip in the launch video, wouldn’t you? Because <you> want us to believe that the world is free of foreplay and full of climaxes.
Be careful friends, do not confound knowledge with learning, verification with truth, and <you> with you. Do things that don’t scale, and we’ll see each other again in this beautiful garden of forking paths.
