Welcome back to the AI For Humans newsletter!

To start, a personal confession: As a student, I was pretty lazy.

I was able to get through school without having to do a ton of work. I didn’t get straight As but I was able to keep myself in the A-/B+ range fairly easily. I’m not proud of this fact now and, had I realized what I might’ve been able to actually accomplish, had I put my mind to it, who knows what I’d be doing. Maybe still writing this newsletter.

The secret to my coasting success was my ability to convincingly make arguments, both in papers and in person, without having a lot of knowledge about the subject matter. I made sure I knew just enough to sound convincing. I was an amazing bullsh*tter.

And, it turns out, LLMs are really good at this too.

In May of this year, the New York Times ran a story that got a ton of pick-up discussing how as AI advances further and further and, ostensibly, gets smarter, these large language model systems are hallucinating more not less.

This problem has plagued LLMs for some time and been noted by prominent haters as the reason that that LLMs alone will not lead us to AGI or Super Intelligence.

After all, if you can’t trust an AI system to give you the right answer on a simple question, how are you going to trust it to give you advice on your business? And this only compounds in the world where AI agents go off and do a bunch of work for us without oversight.

Enter OpenAI’s new hallucinations research.

Turns out, LLMs Do Better When They Say “I Don’t Know”

Here’s the big takeaway: hallucinations aren’t some weird glitch. They’re a product of how these models are trained. Right now, LLMs get rewarded for sounding smooth and confident, not for being correct. Which means they’re literally incentivized to guess, even when they have no idea.

Sound familiar? They’re incentivized to act like I did as a student. To bullsh*t.

The problem is built into the benchmarks. We’ve been grading them on style over substance, so the better they sound, the more likely they are to be rewarded, even if they’re just making things up.

OpenAI’s proposed fix is surprisingly obvious: stop punishing models for saying “I don’t know.” Instead, reward them for it. When you make honesty part of the training loop, the hallucinations drop and the answers get a lot more reliable.

The Potential Creative Downsides of ‘I Don’t Know’

One thing that’s important (to me, at least) is that one of the true upsides of hallucinations is that these models can be a bit more creative. I realize I’m unusual but I’m always a little impressed at the story the LLM tells me when it’s obviously making something up.

I have a weird hunch that hallucinations might be key to AI becoming more creative and less rational but I understand the market has very little interest in that.

In our work on AndThen (my start-up in the AI audio space), we’re very interested in how these models can push the story forward & surprise our users. Normal businesses, and to be clear, these large AI companies that depend on businesses for revenue, don’t want surprise.

As I mentioned last week, what they’re looking for is clear, actionable results.

My hope is that we get both.

After all, according to Anthropic’s Dario Amodei we’re in for quite the ride…

That’s it for today. See you on Friday for the podcast!

- Gavin (and Kevin)

THIS WEEK: Salesforce announced it cut 4k jobs thanks to AI. Save yours! 👇

We 💛 This: The Charlie & Lola TikTok Trend

Google Gemini’s Nano Banana AI image model keeps surprising us but, as you might imagine, it’s also driving some significant trends on social media.

This week, we’ve got the ‘Charlie & Lola’ trend:

The prompt is actually really simple & super fun. While it’s clear I’m WAY too old to be doing this particular prompt (I’m pretty sure Charlie & Lola was something my daughters watched), I have debased myself below for your entertainment.

PS, this original photo is from Perm Week which, yes, we actually did at Late Night with Jimmy Fallon. Let me know if you think I need to go back to this look.

Prompt is below: