Earlier this summer, rangers with Shasta-Trinity National Forest found a woman camping in an off-limits area on the south side of Shasta clearly marked with “no camping” signage, Meyers said. She told rangers she had asked ChatGPT where she could camp in the forest.
“AI told me I can camp here,” she said, according to Meyers.
Then in August, while hiking in Shasta’s popular Bunny Flat area, a man stopped rangers to ask for directions. He showed them a print-out map he’d been following and explained that he’d prompted ChatGPT to produce it for him.
“It was terrible — terrible — not even close to accurate,” Meyers said. “It had route lines and place names that were totally off — just ridiculous. We were like, dude, you gotta throw that away.”
From the latest mishap:
During the hike down, the young climbers revealed to rangers that they’d relied on ChatGPT to guide their planning. Authorities noted that as a “key factor” of their mishap.
The climbers “reported that AI recommended simple carbohydrates for their climb, rather than fats because they ‘take too long to digest,’” according to the official report. “In their words, ‘We relied too much on AI rather than our own critical thinking.’”
Edit - from the non-Archive version of the article:
Correction: A previous version of the story misstated the AI tool the climbers used to plan their trip. It was Google Gemini.
The whole situation with LLMs is so incredibly infuriating. The technology itself is genuinely cool. But the fucking greedy tech bros make it actively worse by training it to pretend it was some autonomous artificial intelligence that it so clearly is not. And due to all the consequences from capitalism milking the shit out of it the whole fucking society goes to shit and now you are left with basically two factions: on the one hand you have the evangelicals that honestly think that LLMs are the best thing since sliced bread and it will revolutionize the world and on the other hand you have the haters that think LLMs are a total scam and just as bad as any other autocomplete, which has been around for decades.
The truth is somewhere in the middle. The tech is actually pretty impressive but the tech companies are overselling it hard and train the LLMs to oversell themselves hard to impress investors and the public. This is the real reason for all the AI psychosis shit. You cannot blame gullible people for being gullible when the tech is actively designed to trick people. Don’t blame the people, don’t blame the tech, blame the fucking assholes that are responsible: The greedy fucking tech bros.
I personally use LLMs pretty much every day and i have personally also used it to plan trips. But i do know that it makes shit up all the time and i have to sift through the annoying sycophancy and useless rambling to get to the actual useful bits: A very efficient way to search for information that i know is probably widely available in books and the internet but would take me much more time to find manually.
I seriously think that this technology could be so much more useful if it was optimized for those kinds of tasks and would not pretend to be a sentient artificial intelligence all the time.