Local Models vs Hosted Models: The Wrong Debate

There's a trend right now of models continuing to get better. Companies are releasing better and better models. There are closed models, there are open weight models, and now there are really small models that anyone can just download and run on their own machine. And they're getting really good.

So there's a debate about whether you should use these local models or use the hosted models and pay per token.

You Can Get a Lot Done Locally

The reality is that you might be able to get lots of work done as a developer using a local model. For plenty of everyday tasks, a model running on your laptop is more than enough.

The Factory Analogy

Using a local model as a software developer or a knowledge worker is akin to a factory worker using some kind of automated machinery. Without it, they can't really put the device together by hand, or it would take hundreds of hours with screwdrivers. With the machinery, they can do it in a minute.

That's a human augmented process. And it's a big improvement.

But what's happening is that other companies are building fully autonomous versions of that. If you can build a factory that runs on its own without any humans, you're going to get 100x productivity versus the human augmented one.

It's the Same With Models

Eventually you're just going to have AI staffing, which can just do the job, versus humans who want to use a local model, a hosted model, or any model.

If a human is driving the decisions and is involved in the production of the code, or the content, or whatever it is that they do, then they're never going to be able to keep up with the AI.

The Inevitability

Of course, there are other fields where AI is not fully autonomous today, and it's hard to see how they will be. But that solution is an inevitability.

So the question isn't local versus hosted. The question is whether a human needs to be using models at all.