You Cant Tell When Its AI Content And It Might Not Matter

I hesitate to write about AI often because I think it’s too profound a shift in the way tasks get done, reviewed, scoped, or incepted into one’s mind to call it just “good” or "bad.”

However: I do have thoughts – many. And a lot of them are ones I don't hear people making very often.

Here are 5.

You Can’t Tell When You Can’t Tell

During World War II, the Statistical Research Group at Columbia University examined planes that returned from combat to see if there was a scientific way to know which parts of future planes to reinforce going forward, in order to make them safer for pilots. It resulted in this image.

Wikimedia commons Martin Grandjean (vector), McGeddon (picture)

I think about this a lot when I hear people talking about how they can “always tell” when something is written by AI.

The obvious conclusion to draw from the image is that areas over the wings and tail of the planes needed to be reinforced in order to prevent bullet damage. The problem with obvious conclusions is that they’re often wrong.

Survivorship bias is what you get when you draw conclusions based on evidence that “survived” a selection process you overlooked. The planes that made it back to Columbia all made it back to Columbia just fine. Therefore, the better solution to the problem was to reinforce the areas of the selected planes with no bullet holes because planes hit in those areas never made it back.

When people talk about how they can “always tell when something was written by AI,” that just means that they can “always tell” when something was poorly written by AI. 

And honestly (see what I did there?)?

The fact of the matter is you can’t tell when you can’t tell. The allies had data on how many planes of theirs never came home, but we don’t have data on how often content is written by thoughtful partnerships between LLMs and human copywriters. 

We need to walk mindfully forward into the future because planes exist, they help us get places faster, and there were people who railed against the advent of aviation because they were invested in trains. But a century-and-a-quarter after the Wright brothers’ first flight, most of us would be hard-pressed to name who those people were in the first place. AI exists, and we need to learn to live in that reality.

Which brings me to my next take:

It Might Not Matter If You Can’t Tell

Our relationships to technology change with its utility. Our values change by the generation.

A recent quiz held by the New York Times polled over 86,000 participants in a blind test and found that 54% preferred passages written by AI to ones written by human beings. That suggests that writing quality of an average LLM may now equal that of a professional copywriter. 

That’s obviously scary if you’re a copywriter today. But I think the smartest thing to do is to look to the future.

In late 2021, Epic Games created an interactive gaming experience to show off its new “Unreal Engine 5.” As part of the promotion, Keanu Reeves and Carrie Ann Moss sat down with The Verge to talk about The Matrix Movies, the fourth of which released around the same time.

Around the 7:30 mark Keanu recounts a story in which he had to explain the plot of the original 1999 movie to a group of Gen Alpha teenagers at a dinner party. “It’s about this guy and he really wants to know if it’s all real.” 

The kids: “Why? Who cares if it’s real?”

The Verge

They say the latter generation judges the former on beliefs, biases, and habits that never would have occurred to the elders to question.

Gens Alpha and Z are about to enter their prime earning years.

And if their collective shrug about artificial intelligence is as widespread as the Matrix case-study may suggest, then entire arguments – over how to label AI-generated content, over who or what actually does the copy writing of the future, over the merits or demerits of integrating it into any workflow – may be moot.

The future belongs to the ones making decisions in the future. It’s time we all started listening to them.

Speed ≠ Better

This argument is well-trod, so I’ll be brief. 

Prevailing sentiment among the executive class is that AI is a shortcut to higher productivity because it speeds up individual tasks. Prevailing sentiment among boots-on-the-ground individual contributors is that speed does not equate to better performance, and that noise will not cut through more noise.

Both are true. Understand the function of specific tools. Let’s move on.

Speed ≠ Worse

Raise your hand if you ever shipped something you thought was not your best work, but “good enough.” 

Keep your hand raised if that thing ever did its job anyway.

Head-to-head with your Brené Brown quote of choice, the phrase "Good enough is literally good enough," will lose 100 out of 100 poetry contests, but it is something worth considering if your workplace still operates under a “move fast and break things” model.

Oftentimes it’s better to ship something and iterate than spend any more time than necessary ideating what the perfect thing could be. That’s an argument in favor of more for the sake of more.

AI-enhanced workflows will speed things up and sometimes create noise. Not all noise is bad. It can get better in the patch release.

The Best Argument Against the Development of AGI Is That We Would Abuse It

Credit for this broader idea goes to a guest whose name I can’t remember from The Ezra Klein Show years ago. (The LLM I had proofread this blog post asked if I wanted its help sourcing them but, that somehow felt wrong.) The examples and the specifics are my own.

A 2021 amendment to the UK’s Animal Welfare (Sentience) Bill classified lobsters, octopuses, and crabs as sentient beings. Humans eat them.

In 1930, John Maynard Keynes wrote an essay titled Economic Possibilities for our Grandchildren in which he predicted that by 2030 we’d work 15 hour weeks because of machine-aided increases in productivity. We don't.

Climate science is a 50+ year old field of study. The world is still on track to exceed the 1.5ºC Paris Climate Agreement threshold, despite evidence that higher temperatures will lead to drought, flooding, and greater risk of food insecurity.

We inflict suffering because we choose to.

...

As far as we know, LLMs do not think. But if humanity some day in the near future births a truly “intelligent” Artificial General Intelligence, history will record that they were a big step in that direction. 

And if that day comes, we’ll need to deal with moral ramifications of introducing a new life form, while our species has the track record that it has.

I’m not making arguments one way or another here. Fundamentally, the point of a society is that we pool our resources and strength in order to do things together none of us could do alone. There are still cases to be made for continuing to put calamari on our plates, working 40 hours as a way to add meaning to our lives, and that burning oil has led to more development in the last 150 years than in the prior 150,000.

Most people don’t wake up in the morning dreaming of ways to make things worse. But we do need to be clear about what we're doing.

I’m Not Here to Moralize, But I Am Here to Observe

That’s the first step in storytelling – understand the story. LinkedIn, the average work-conference, the watercooler. These are all places where there exists some sort of prevailing wisdom.

AI content is everywhere now. The arguments about it are louder than ever. And most of them are drawing conclusions from the planes that came back.

Let just make sure we look at the whole plane.