Hey Reader,

People keep asking me about open source models, especially now with GLM 5.2.

But here's how I think about them:

If you think hiring a pro is expensive, try an amateur.

Sure, they can be cheaper, but I'd never run my business on them.

Here's why.

What local models get right.

First, they're cheap. They sometimes run on your own computer.

So you don't have to pay for every question you ask throughout the day.

Second, they're also private. Your data never leaves your machine.

A local model is like a owning a full building where nobody else can get in.

With cloud AI models it's more like renting an apartment where the landlord has a key and can show up unannounced.

And thirdly, they can be fast. But that depends on the model and how smart it is.

Those are real benefits. So why do I don't I use them?

Too good to be true.

The first problem is benchmaxing.

Chinese labs pre-train these models to beat the exact tests everyone uses to rank them.

It's like interviewing a new hire and them absolutely acing your questions, but once the real work starts in the office, you realize their CV was full of lies.

The second problem however, is bigger.

None of them actually keep up.

Every open model is 20 to 30% behind the best one.

And yes, that even includes GLM-5.2, which big names like the CEO of Vercel say is almost Claude Opus level.

I've actually tested it for a few days, and for most tasks it's pretty good.

Sometimes it even beats Opus 4.8.

But if you give it something much harder, it gets stuck fast.

It also doesn't know as much about copywriting or other general intelligence.

GLM 5.2, like other open models, often spins in circles, thinking and re-thinking for a very long time.

The state-of-the-art models used to do this, but they've long overcome it.

The question I ask instead.

I stopped asking which model is cheapest.

Instead, I ask one thing only:

What am I building, and what is it worth to my business?

Then I pick the best model for that job. Not the cheapest.

Because say the cheap model saves me $1,000/mo, but costs me 10 extra hours to get the same result.

That values my time at $100/h.

Everyone values their time differently, but just to put things in perspective:

I don't typically do 1:1 coaching, but when I do, the investment is $1,500/h.

So the math just isn't mathing.

When the cheap one still wins.

Three times I'd maybe grab a cheap or open model on purpose.

First, it's a great way to safe money. GLM 5.2 is about a quarter of the cost of Opus.

Second, data privacy. Ran locally or on a ZDR provider, there is unconditional privacy.

Thirdly, they can run offline. Which is cool, but still requires massive hardware for powerful models.

What's the one model you'd actually trust your business with?

Talk next week,
Rob