Showed up first on Lobste.rs (by 42 min by my stats) with the author themselves as a submitter:

https://lobste.rs/s/klw6bu

most of the discussion is about whether they’re a spammer or a promptfondler.

HN seems more enthusiastic

“In fiction, the main character is often the centre of events,” he says. “The problem is that, sometimes, AI can actually get mixed up about which idea is a fiction and which a reality. So the user might think that they’re having a serious conversation about real life while the AI starts to treat that person’s life as if it’s the plot of a novel.”

This is such a bad way of explaining it. Yikes. The issue is that it’s all improv roleplay, all text is bullshit in the frankfertian sense. It might be accurate — lots of fantasy contains true facts like chairs and tables being made of wood and used for sitting at — but it might be completely divorced from reality and the model cannot know!

man if there was a common household chemical, like plastifier, that causes psychosis after couple months of use in small segment of population, you would absolutely hear no end of this [1]. but you see, now it’s an app, oh we’re just throwing up hands, can’t do anything about it and it’s but one of inconveniences that we all have to pay for Progress and Glorious Technocratic Future

[1] i’m thinking here of things like phthalates, bisphenol A, PFOA or paraquat, herbicide linked to increased risk of parkinsons and banned in some countries. their harmful effects need a lot of time to show up and are nowhere as dramatic

Both the rationalists and chatbots in delusion mode offer people the chance to be the main characters in a story and not just part of a team. Organizing lawyers and lobbyists to make chatbot companies’ lives hell over years is much more effective than firebombing an office or stalking a random vehicle which the bot says has a team of corporate assassins inside.

Taka in Japan is a neurologist which I think is a type of MD.

I don’t want to underplay how bad this is, but did BBC really need to use the “slutty anime witch” image of Ani for that story? Or was that the actual avatar he had set for it? Like, I’m not saying that it changes the problem or makes him less the victim here but it is yet another example of “goddamn why is this cyberpunk dystopia so cringe?”

A few details that remind us of the horrors we are skirting around. Imprimis:

(The landlord’s friend Thomas) Young said he searched the property after the attack and found used surgical equipment, more than a dozen laptops, and expensive electronics stashed inside the cargo trucks where the alleged assailants lived, which were registered in Vermont.

“It was actually very uncomfortable,” Young said about walking into the trucks. “You kinda wanted to put on a hazmat suit before going into it. It was really just creepy in the extreme.”

II. Another local story about a court appearance.

(A Zizian’s lawyer) Stelzig raised specific concerns that Youngblut’s confidential medical records would be incorrectly opened as part of the case. “Our client was shot and is still receiving frequent medical treatment” in jail, she said, referring to injuries Youngblut sustained in the shootout.

III. Ziz Lasota asked a rhetorical question about what might have happened to missing people who got too close to her:

Did I drive them to suicide by whistling komm susser tod (sic)? Maybe they died in a series of experimental brain surgeries that I performed without anesthetic since that’s against my religion, in an improvised medical facility? (Evan Ratliff, Wired)

The medical records might just mention trans stuff or an Asperger’s diagnosis. Sneering has not felt adequate for a while but I don’t know what else to write.

Edit/ Zajko, who wrote “I’ve never seen (Ziz) do an evil thing,” also wrote that Ziz told her to murder an associate or Ziz would murder her. Our intuitions about what went on in those box trucks and on the boats are unreliable and the reality may have been worse than sleep deprivation and paranoid ramblings.

Also a year ago, Maximilian Bentley Snyder the Zizian dictated a 1,500 word manifesto telling Yudkowsky to become a vegan. These have not been published because murder cult but three of the Zizians have issued statements leaving just Ziz, Youngblut, Suri Dao, and Alexander Jeffrey Leatham alive but silent. Two died after attacking outsiders, and two took their own lives, so I count 11 known associates.

that’s a horrifying situation to be in… good on the community who originally cancelled his show for apologizing

If you asked me to guess the kind of kerfuffle that might develop between a Cape Breton fiddler and AI, I would have answered, well, my entire knowledge of Cape Breton fiddling is based on the paper “Cape Breton Fiddling and Intellectual Property Rights”, so my guess would be just “the normal AI stuff”. And I’d be totally wrong and reminded that just because I know one thing about something doesn’t mean it’s the only thing.

a world where lots of matter and energy was spent on its weird and alien needs, rather than on human beings staying alive and happy and free

Good thing we don’t live in a world like that right guys?

Wow, that’s probably one of the most in-depth critiques of the book I’ve read. Kudos to the OP

a leaderless cult had freshly formed

a Stand Alone Complex, but with slop

Well, we do have computer science, so necessarily we must have computer religion/superstition

I like Evans’ take that since there’s bound to be oodles of cult related literature and interactions and also tons of self help and guru stuff in the training datasets, it stands to reason that if you interact with a chatbot in a way that indicates vulnerability to these things there’s a considerable chance that it will decide the expected response is to prey on you.

Also Scott Aaronson jump scare near the beginning, apparently he was blurbed for something.

From your prev post:

There is a “lattice” which connects all consciousnesses

The noosphere, the old cosmists strike again. This sort of stuff and the global consciousness projects (who used random number generators iirc) etc are def part of the training data.

Jesus his fucking hat metastasized

Clown v. Clown. This is about the level of discourse Yud deserves.

It’s absolutely crazy, but I think Yud is the less unhinged person here

late but I didn’t watch the video (busy) and was under the initial impression that Yud changed into the more silly outfit after the bro revealed himself to be bullshitting, but no he just…showed up to the debate in sparkly goggles and a sparkly hat. Before he knew he was debating a clown. For reasons i cannot possibly comprehend.

Yeah, the comparison to a metal lathe (I’m assuming that’s what is meant by “machine lathe”) irks me. Metal lathes are useful tools that come in a variety of sizes, can be operated independently of their manufacturer’s wishes, and on the bigger ones, a sufficient lack of respect will kill you and it will hurt the entire time you are dying.

I wouldn’t compare llms to any sort of useful physical tool. I don’t have to plead with my screwdriver to not cam out screw heads, I just need to be cognizant of how I am using it. I don’t have to beg my hammer “please don’t hit my thumb”, I just need to not be an ape/buzzed while swinging it. I respect physical tools and their use in part because they do work and in part because improper operation will cause you to have a bad time. Nobody at Estwing has publicly said “well mate, we’re going to capture all your hammering and rent it back to you”. Nobody at Bridgeport gleefully reported that their tooling will cause massive unemployment because it is so good.

I do not respect LLMs because LLMs might still kill you but only in the stupidest way possible, and because their main proponents have no respect for anything and this has been made painfully obvious over and over again. Would using them allow me to craft better sneers about them? Perhaps, but I shouldn’t need to do that because the people at the top of these things are evil and while that alone should be enough, their biggest boosters are credulous idiots and many of them were already awful well before we were playing enterprise pretend on the scale of billions of dollars.

In conclusion, I would ask software people to stop comparing shit software to useful tools.

My dominant MtG colors are blue and black.

Basically admitting to being evil

Yud:

I didn’t stick to merely the culture I was raised in, because that wasn’t what that culture said to do. The characters I read didn’t keep to the way they were raised. They were constantly being challenged with new ideas and often modified or partially rejected those ideas in the course of absorbing them.

Also Yud: ewww Neuromancer is icky

Yud:

But if you consider me to be more than usually intellectually productive for an average Ashkenazic genius in the modern generation

It’s not just a load-bearing if, it’s a conditional that manages to be vaguely racist under all the smug. C-c-combo move!

I feel like a lot of my writing on rationality would be a lot more popular if I could go back in time to the 1960s and present it there. “Twelve Virtues of Rationality” is what people could’ve been reading instead of Heinlein’s Stranger in a Strange Land

This is someone nakedly fantasizing about being L. Ron Hubbard.

neuromancer is brilliant prose first and foremost, and yudkowsky not being able to realise this is so very symptomatic