I’m not an AI techie and I’m not well educated on LLMs/AI in general. The issue I’m having is the dichotomy between how people talk about it. I’ve heard completely opposite ends of the spectrum. From AI is quickly becoming advanced enough to threaten human society, all the way to its a glorified dictionary/search engine that can’t really do much of anything. I know Lemmy has a steep anti-AI slant. But, can someone talk about this in a way I can wrap my head around why the divide seems so big?
The people claiming it’s humanity ending are all directly involved in the AI companies. One take is that they are the ones that know the most about it.
My take is they are financially incentivized in making sure their investments grow in value.
A third take is that they are unhinged cult members who are high on their own supply. Although I expect it’s a mix of both types (maybe within the same people).
Cory Doctorow also thinks that it is a coalition of true believers high on their own supply, and cynical grifters collaborating with them. https://pluralistic.net/2026/09/17/porque-no-los-dos/#fascist-incoherence
Many are skeptical that there is a push by ai companies to make them appear more than they are due to their weak economic footing. From my experience with the llms I am not frightened they will become self aware. I am frightened that we are wasting energy and resources while polluting the planet further for little return and it will result in messing up our economies even more.
Personally, this just feels like marketing and these big asshole companies trying to resurrect the 2025 level of hype in order to salvage their IPO’s.
If they honestly believed this I can’t imagine that their workers wouldn’t do something more drastic.
I’ve heard a few explanations for the “AI will kill us all” narrative.
-
AI techies are in a cult. They genuinely believe this stuff because they’re crazy.
-
It makes the AI sound powerful and impressive, which is ultimately good for the AI companies.
-
They’re starting to face competition from open-weight models, so they want to scare politicians into crafting regulations. They think that they will be able to influence the regulations to favor them and squeeze out their competition.
And, of course, there’s
- It keeps people talking about AI. Because whatever you do, don’t quit talking about AI, because if you quit talking about it, the line might go down! And the line going down is very, very bad.
Okay cool, thanks!
-
If the techies believe it and they are the “experts” what makes them crazy? Wouldn’t their opinion be the most informed?
-
Yeah that part definitely makes sense
-
That definitely makes sense from a capitalist perspective. But, is the argument against regulations that they would be bad for the concentration of power it would give select companies? Or just that they aren’t necessary because the tool is of little use so it’s just wasted time/money.
-
That makes sense but trillions of dollars are being spent. Anything that amount of money is sinking into is gonna be talked about a lot. Especially thousands of jobs cut because of it.
If the techies believe it and they are the “experts” what makes them crazy? Wouldn’t their opinion be the most informed?
Listening to them speak about it, no. They’re morons. They clearly didn’t make the tech. Workers made AI. Just like they make everything. These freaks make nothing. They can’t give single coherent explanation of how LLMs work, how they could “kill” us, nor do they seem to know how or what “guardrails” need to be put in place.
But if they really are a cult then we all know what the next step is. They need to STFU and get off of our planet.
That’s what I mean though, the subject of this article was a 27 year old worker. Not some high level Sam Altman type.
- No, their opinion would not be the most informed. They are biased in favor of making money. They know they can’t deliver on strong AI, but they’d be happy to milk your fears.
I’m not talking about execs.
-
-
‘Sensationalist hooey, cooked by superstitious yokels seeing phantoms of their own imagining.’
In addition to what everyone else said, I’ll tell you why I don’t think that what we have currently can become some runaway superhuman intelligence:
All “AI” people are talking about are basically “just” LLM wrappers: tools built to run a LLM in a loop while giving it access to the ability to write and run code. This gets pretty powerful, since you can do a lot programmatically (including monetary transactions), but has a fundamental limit:
LLMs are trained ahead of time. And that’s a very cost intensive, long process. Beyond that, the LLM has only a context window of finite length. So there are no long-term planning capabilities. While running after being trained, the LLM can only condense knowledge into a summary and then re-read it, a lossy process that might introduce mistakes in “understanding” and will make any prolonged task degrade over time.
And even if they figure out how to make the things actually self-modify and learn things, you can’t give every session that ability, since that’ll vastly increase the storage and performance requirements. Do sessions with that ability will be closely monitored and limited.
So I don’t see any space where an evil AI could emerge, even when factoring in an essential possible future improvement to the technology.
I’m not worried about AGI. I’m worried about the insane military applications they’re developing right now, and giving Artificial Incompetence control.
I watched a grokbot get confused connect to the wrong vnc console and try to shut down its own little computer.
I’m not worried about AI taking over the planet. I’m worried about it clicking the wrong button with no oversight
Yes, but when the shit hits the fan and they blame the AI, we shouldn’t let them get away with it. It’s their fault, not the computer’s.
There’s blood on their hands. Either they keep the hype up or go to jail regardless of any of their statements being true or not.
Here’s an idea: don’t use unsafe software where safety is needed.
What makes it unsafe?
To me, what makes LLMs unsafe is they are nondeteramistic.
If I remember correctly; Nondetermanistic means the LLM can, could, and will have different answers/outcomes for the same question.
The lack of consistency potentially leads to negative outcomes?
Well, if the output varies in a way you can’t predict, then even if one of the inputs is “don’t do anything negative” it might not always follow it exactly.
Makes sense
Yeah, because it’s all statistical. You can test this yourself. Go to Google and search for something. Then search for that same exact thing again. Gemini will give you a different response each time.
You don’t know? … Wow. How about the error rate? Let’s start there.
No, as previously stated I am not educated on the subject almost at all. Why I framed the post as a question.
Humanity should be way more frightened of greedy psychopaths running the world than any technology.
And they’re peddling “AI” so aren’t they married in this case?
A lot of us here hate streaming platforms because they’re designed to benefit greedy capitalists first and foremost. That doesn’t mean the core idea or the technology behind streaming platforms is bad, though. An open source project like Jellyfin that is designed purely to benefit the end user is something that legitimately enriches all of our lives.
Similarly, just because scumbag companies like Anthropic and OpenAI are plundering the world for their own deranged benefit doesn’t make the underlying idea or technology bad. The tech is built on contributions from a lot of brilliant people over a long period of time whose names are not Sam Altman or Dario Amodei. Open LLMs that run on gaming rigs are quickly catching up with cloud-based models as well.
At this point, I’m running Qwen 3.8 27B on a gaming GPU that has similar capabilities to Claude Sonnet 5 without giving Sam or Dario my data or $200 a month, just like I run Jellyfin without giving Netflix or Disney my money or data.
I feel like the AI hype has died and people are trying to prop it up with doomsday scaring.
I have no idea if that’s true.
Their product hasn’t done anything new or different in a while. They keep coming out with a bunch of scary hype stories every day so it all seems fake as hell now.
Yeah I can’t tell if an article like this has the intent to spread a fact, enforce anti AI attitudes or enforce pro AI attitudes. Realistically all three are possible but I think one of the latter two are most likely.
It’s to convince congress to outlaw local models.
Why? Guessing that makes it easier for a monopoly?
Technically not a monopoly, because that would mean only one company has control, and that’s illegal. It would be a small number of companies, though.
Still problematic.
I knew a guy in high school that told girls they shouldn’t date him because his dick was so big it would hurt them. You can wax philosophical on his motives if you’d like, but spoiler alert - his dick wasn’t big.
That’s exactly what I think is happening. Remember a few months ago anthropic saying mythos is too dangerous to release until the government as the idiots they are starting believing them and banned them from exporting it, therefore losing them money. Very quickly they came up with fable (ironic name isn’t it) a week or two later saying it’s got the proper safeguards now. Now they’re all trying this marketing strategy asking the gov to regulate them, seemingly because they’re dangerous and not because they’re getting diminishing returns and feel threatened by the chinese competition
Good job in helping with their guerilla marketing campaign.
When a corporation realises that working for government will be more profitable than working for consumers, they pivot to scaremongering, but always with the undercurrent of “and that’s why you should PAY US so that you own the nukes, and not your enemies”
I don’t have the confidence in my knowledge/understanding of the topic to just take a “its all bullshit” view. Especially when many of the people who believe that act like you. You don’t have to be a prick to explain something or talk about it.
LLMs basically work like auto-correct. They’re much more sophisticated, but it’s the same basic principle. Instead of “Which letter is most likely to follow after this series of letters?” it’s “Which token is most likely to follow after this series of tokens?” A token can be anything, but usually it’s a word.
Now, given that, consider a statement like
We really do earnestly believe AI could kill all humans!
Okay, how? A statistical model generating tokens doesn’t have thought, it doesn’t have emotions, so it can’t want to do something, much less make plans and carry them out.
Could AI give somebody the ability to kill all humans? How? Under what scenario? Is somebody putting an LLM in charge of their nuclear stockpile? You don’t blame the LLM. You blame the humans who made that decision.
Okay that all makes sense. Appreciate you taking the time to type it out.
They are lying about their fear. That’s all they can do. Stop believing any claim of theirs. Stop being manipulated.
They are snake oil salesman. They sell vaporware. Don’t fucking buy it.
I’m more worried about the people using AI than the tech itself.
Every issue has been because corporations have “get out of jail free cards” these days, greed is sky high, and AI psychosis is rampant.
All those supposed AI agent hacks happened because someone set that as a task for the AI agents. Yes, someone thought it was ok to do so for marketing and profit. That’s the problem, companies are willing to use the AI tool to commit crimes.
Imagine if weapons companies advertised their products by shooting people in the streets. Someone thought it was a good idea, and can get away with it.
I don’t believe at all that AI agents have free will enough to decide to hack companies for fun. They’re just glorified text generation machines hooked up to an interface. They don’t have intentionality unless some human gave them the directive and wrote them a large cheque.
Also, we know all these AI company billionaires read dystopian scifi novels like business strategy textbooks. That’s why everything surrounding AI is so overtly scifi coded.
it’s the former juuuuuuuust until the latter
once it escapes the lab there’s no putting it back in the bottle
Unlike a bio-virus that has evolved for a billion years across all the niches of our ecosystems, when AI “escapes the lab” it’s entirely confined to man-made infrastructure, it’s not going to infect a flock of migrating birds and hop to the other hemisphere, it’s going to be in systems we built, mostly within the past 30 years or less, mostly in the past 30 months for the powerful ones.
When AI starts demonstrating “copying itself” through DNA encoding infecting living beings without killing them (too quickly), and/or “running” on something more ubiquitous than massive data-center hardware, that’s when it’s over.
It may be confined to man-made infrastructure, but that same infrastructure is what keep billions of people from starving to death. At this point, computers and networked systems are so baked in to every facet of our logistics chains and government that simply shutting it all down isn’t an option. The nightmare scenario is where rogue AI escapes the lab and copies itself to so many systems that the only way to kill it is to shut down every computer on Earth.
That’s the thing, at least right now, AI is dependent on new data centers, excess capacity built just for it - it’s not going to “escape” to the banking system and the tractor navigation chips, if it could do that they wouldn’t need to be building all this capacity for it.
Ed Zitron’s podcast “Better Offline” recently talked about this. There is a small tech community called the Rationalists who believe AI super intelligence is inevitable and it will either accelerate humanity to new heights or kill off the species. These rationalists are actually a very prominent faction within tech and the AI industry. Apparently the language used within top AI firms is largely about the immense danger posed by AI super intelligence if any other group or company were to achieve it first. This creates an echo chamber that makes the people in the industry kind of lose their mind and misinterpret the capability of LLMs.
Everyone outside that faction recognizes that LLMs are just unreliable cloud software with limited use cases and that there’s no evidence LLMs could actually lead to any kind of intelligence.
Edit: Please, someone stop me from commenting on an AI post next time I’m tempted. I’m so tired of people replying, arguing with opinions I don’t have, based entirely around a hypothetical technology that doesn’t exist yet. LLMs make slop. Thats it. They hacked a company after weeks of 100s of instances being run, using old exploits published online. San Francisco will be underwater from climate change before LLMs can do anything more.
Aren’t they called TESCREALists or something? It’s like a mish-mash of pop culture and pseudo intellectual BS
My Problem is: Even if your take on AI is true, we give an unreliable piece of software access to the internet and more and more systems and infrastructure. Even if it is not a malicious AI, it can be so unreliable, it might kill us by incompetence. a Sme can be said about some presidents…
LLMs are not intelligent. They lack morals, knowledge, and intent. The only remote risk of extermination is if a company with absolute impunity and endless resources told an LLM to hack the nuclear weapons and if knowledge of how to do that was already published online and processed as training data.
LLMs cannot do anything new, they can only regurgitate information thats already been stolen.
Or if the military put an LLM in charge of nuclear weapons. But surely nobody would be stupid enough to do that!
You’re confusing morals, knowledge, intent, and intelligence with capability. An atomic bomb has no will of its own either.
That is simply counter-factual. LLMs can absolutely do things that have not been published before, and they can access knowledge that is not in their training data through RAG. Intelligence is debatable, but reasoning models can reliably imitate “thinking” to a degree, and recursive self-improvement might lead to an autonomous advancement of intelligence. That LLMs allegedly “only regurgitate information” is simply wrong.
any kind of intelligence
A photocell that turns on a streetlamp after sundown, or when a dark cloud passes over the sun, is a kind of intelligence.
Got it. So plants are intelligent.
