It wasn’t. But that doesn’t mean we shouldn’t strive to make it be the best version of itself it could be, and fight tooth and nail against attempts to make it worse.

You’re probably burning more energy turning it off and on again. It doesn’t really use any noticeable power sitting idle.

I am absolutely not burning more energy than a frontier model by doing things like putting my laptop to sleep or shutting down unused services when I want to conserve battery power.

Anyway, a direct comparison would be pretty difficult because your model is probably tens of billions of parameters, not over a trillion.

True.

Energy consumption per output token will probably be a bit higher for the frontier models but something that people have found is that higher quality models often need fewer tokens to achieve the same goal.

That’s actually not true. In fact it’s much the opposite. Frontier models churn through tokens at a much higher rate, because of their higher complexity and higher number of parameters. Research is still new on this, but having a frontier model analyze your code files versus a small, local model for the same task seems to be enormously wasteful. If you must use a frontier model for something, have it do that work after receiving the output from an agent using a small model to read and summarize your code.

Plus how many times do you re-prompt your local model vs Claude Fable or Opus for example to get the desired result?

…Almost never? I’m not a fan of letting AI do much of ANY of my coding, because it will inevitably bloat my codebase with garbage regardless of which model I use. So I severely restrict my model usage to simple, clearly-defined, narrow-scoped tasks that can save me a bit of time, and that’s it. With guardrails and discipline like that, I barely ever have the need to re-prompt.

Are you serious?

• Local model: Spends most of its time turned off. Only active when I want it to be active, and only for a little while. Dedicated solely to generating the small amounts of code I use it for. Does nothing else. Costs $0 per token, and electricity costs are negligible.
• Frontier model: Always on, running on millions of GPUs. Would be burning down the planet even if hardly anyone was using it. Incredibly wasteful, being used for trivial tasks and convincing people that their horrible ideas are visionary every day. Misspelling “strawberry” for the masses. Trained specifically to be addictive. Can easily cost a software developer who is addicted to AI thousands of dollars a month, with the recent price increases.

I’d love to see some data to back up the assertion that frontier models are somehow cheaper and more efficient than running a model locally.

No, there really isn’t. The frontier models are created through massive plagiarism. They’re designed to be addictive to use. They consume massive amounts of resources to feed you slop. They are inherently unethical. We’re burning the planet down to keep them running, and we don’t even have a demonstrable financial ROI to show for it.

Stop using them. If your employer makes you use them, maliciously comply by wasting tokens until the financial pain is too great for them to bear and they stop. If you yourself are addicted, switch to small, local, open-source, open-weight models you can run yourself. You won’t burn the world down running a small model on your own computer.

It’s becoming painfully obvious that there is no way to ethically use frontier models powered by these monstrosities. It is currently 100 F in Tuckahoe, the largest city in Henrico County… and they’re asking people to not use electricity so that these heat-and-pollution-generating slop factories can use it instead.

This is insanity.

A bitter lesson in all of this is that the universe isn’t fundamentally fair or just. Sometimes, bastards like Bannon get away with shit and never suffer any consequences for it, and there isn’t even a hell for him to suffer in after he dies on a mountain of booze and ill-gotten gains.

And that’s why it’s up to us - always up to us - to impose fairness and justice on the universe.

It’s also why the price of freedom is vigilance. Somewhere along the way, a huge chunk of us became apathetic. We stopped voting. We stopped giving a shit. Which is exactly what the Bannons, the Trumps, and the rest of those shitheads wanted.

It’s fascinating (in a horrific kind of way) that literally all participants in this war are monstrous. Seriously, there are no good guys. This is turning into WWIII, but I always figured that when it happened, there would be clearly identifiable victims and aggressors.

How naive of me. Everyone involved is victim and aggressor simultaneously.

Yes. They have been for years now. Our far-right are a bunch of fucking idiot assholes, and easily manipulated.

So, uhhh, just put of curiosity, any particular add-ons you’d suggest?

Jesus. We’re gonna have a surgical residency dropout and anti-vaccine nut as our surgeon general.

That’s fucking insane.

If they’re genuinely apologetic and recognize they made a terrible choice, then yes… Welcome back to reality.

invested in his crypto scheme bribed him

Fixed that.

One step closer to just erasing trans people.

I need to remain hopeful. For my sanity.

Genocide. This is blatant, obvious, inexcusable genocide. Israel is now a rogue state.

Actually, it expects them not to, so it can yell about how unfair that is, and the Trump cult will eat it up.

I did exactly that for my mom. Totally non-technical, but she was beginning to absolutely hate all the invasive noise and crap from Windows. All she wanted was to write free of distraction.

So we backed up her files, set up Cinnamon, installed LibreOffice, and imported her files. I set the system up to be offline, since it’s her no-distractions computer, showed her the basics of using it, and basically haven’t heard a peep about it since.

Linux just works, without the bullshit.

If we manage to survive as a democratic republic, there are a lot of fascism enablers who need to be (scrupulously legally) prosecuted for it.

Okay, at first I literally did think this was an Onion headline, it’s so fucking idiotic.

Unlike a lot of people, I think the orange shit-head is more evil than stupid… But goddamn, he’s also incredibly stupid.

Concentration camps. Call them what they are.