You must log in or # to comment.
I give you… “The Grant Money Printing machine!”
Need a grant? Create a disease and submit a paper. Then write a grant asking for money to solve your invented disease.
If you want research grants there is already a glitch for that. You just jam “AI” in your research and suddenly government cares about progress now.
Wait until you hear about paper mills… They were here long before LLMs. This can only get worse… Unless, “we” do something. Or journals themselves do it. Not sure what or how, but better audited ways. Even academia itself could start by valuing more the work of reviewers.
Before anyone shits on these scientists it said over and over again it was made up and that officially the USS Enterprise labs were used to make this discovery.
The Federation would never publish fake data, so it must be true!
Find a way to make AI hurt billionaires and I will support it.
That’s pretty much what local ML is.
If open weights LLMs take off, and business users realize they can just finetune tiny specialized models for stuff, OpenAI is toast. All of Big Tech’s bets are. It’s why they keep fanning the “AGI” lie, and why they’re pushing for regulation so hard, why they’re shoving LLMs where they just don’t fit and harping on safety.
Ok, but who is making those “open weight” models though? Individuals don’t really have the resources to run these huge scraping operations, so they’re often still corporate releases with fake open source branding.
Corporate, for now.
Thing is, once they’re out there, they’re free utilities, and they can’t be taken back.
Also, they don’t really need to aggressively scrape the internet. There are many good public datasets now, and the Chinese are already making excellent use of synthetic dataset generation on (relative) shoestring budgets. Also, several nations and other large organizations are already funding open model efforts, but they just haven’t had the opportunity to catch up yet.
Pretty much is, they’re spending hundreds of billions on a dream (not having to pay workers) that doesn’t work, until they repurpose those datacentres to remove personal computing.
Fortunately datacentres are by design concentrated in space and therefore rather vulnerable.
My friends and I did that in high school. Kinda. We made up new words for “awesome” to get people to start saying it. We started with “bumpenis” like that song is bumpenis. Really we were just getting people to say bum penis. It worked too. We are all just walking talking LLMs.
That’s so fetch!
Stop trying to make fetch happen.
I wonder if we got a group together to go on reddit and stack overflow and give really wrong programming answers and vote them to the top, if Claude would start sucking? They could always just revert to a previous model and it would probably be too hard to get enough people and content to have an effect with such large training sets. Maybe if you use ai? Lol
Didnn’t something similar happen to Grok but ended up with it generating a ton of CSAM material that circulated twitter?
Sorry for being that guy today for you, but you can just say CSAM. It stands for Child Sexual Abuse Material". smh my head :P
Pardon, but what… I did say CSAM, may I ask what exactly you mean?
chain tea, coffee coffee, cream cream.
I imagine this is how it’ll work for stage 2 of Ai enshittifation. They’ll just add a bunch of garbage upstream about a brand or product marketers are paying to push and it’ll infect a bunch of outputs downstream.
https://en.wikipedia.org/wiki/John_Bohannon#Intentionally_misleading_chocolate_study
We did the same before AI. AI is once again just putting an old problem on steroids.
they do the same to protect doctors from malpractice lawsuits. there is a (laughably peer reviewed) study that claims tylenol and morphine are equally effective at pain management.
Good. This shows plainly how LLMs don’t think, don’t truly understand anything, and have no critical ability to do introspection or fact-checking. It seems the only way to teach the world of these things is to make it impossible to ignore via absurd demonstrations like this. If the “AI” well must be poisoned in order to wake people up, I’m all for it.
I don’t see this as a problem, rather, an opportunity to study information & disinformation propogation.
Valid use case, still a significant problem.
But not really a NEW problem. We knew LLM’s are trained on aggregate human data. We know aggregate human data is fundamentally flawed, inconsistent, unreliable, etc.
Like was there a point at which people just decided, nah AI is just plain accurate? Or is that just what morons always thought despite the permanent warnings plastered everywhere saying THIS AI CAN MAKE MISTAKES, CHECK EVERYTHING!
AI’s dont know that birds arent real, or that sometimes the pressure from being under water for an extended period of time can cause fish to explode.
“When the text looks professional and written as a doctor writes, there’s an increase in the hallucination rates,” says Omar.
Huh, now there’s something we have in common. Trying to make sense of something a doctor wrote makes me feel like I’m hallucinating, too. Is there a class in medical school on “Illegible Handwriting,” or is it just a coincidence?
In all seriousness though, I wish I could be surprised by AI failing at this. We have entered the Misinformation Age. There’s no closing Pandora’s Box, though this time I can’t find the “hope” that’s supposed to be in the bottom of it. Society would have to turn real skeptical real fast, but I’ve met enough people to know that such a tranformation is going to take time - and by “time” I mean “decades or longer.” With AI already here, we’d have to wise up immediately… but I fear that humanity isn’t mature enough for that yet.
This is firealarm type stuff that’s what they are meant to do. There has been the alarms like this since gpt-2 they are literally just “here is a sandbox environment, break out and take over the world.” you can easily imagine some crazy asshole out there who would try to use these things for arbitrary evil. So they do it first to see if it is possible. The systems up until now have been famously incompetent so a competent system is genuinely scary.
Wouldn’t humans do the same thing if someone literally writes lies on the internet?
If it were convincing lies made to deceive, then sure. But in this case the papers were deliberately made to be immediately obviously fake, to anyone actually reading them.
So I guess the question would be “would humans do the same thing if someone literally writes obvious jokes on the internet?”
More shockingly, three Indian researchers published a research paper that cited the preprint on the fake disease in Cureus, a peer-reviewed journal published by Springer. It was subsequently retracted.
lol
That’s how we ended up with modern day anti-vaxxers but at least with humans you can strangle the dude responsible. LLMs function like modern idols that the makers use to get away with.
https://en.wikipedia.org/wiki/John_Bohannon#Intentionally_misleading_chocolate_study
Yes, people would exactly do the same, because nobody reads anything but the headline of a paper. Even journalists don’t.
AI didn’t invent the problem, but it put the problem on steroids.
Even journalists don’t
Not sure what point your making here, I wouldn’t expect most journalists to be great at reading the details of papers like this…
Without grounding, correctness is not defined. Hallucination is not a bug that scaling can fix. It is the structural consequence of operating without concepts. – Gregory Coppola
do it again
