This has always been a huge red herring. Llms are built on top of the transformer architecture which does text autocomplete (and yes we can combine text embeddings with other inputs like images). They have some interesting properties where they seem to be able to do text autocomplete in a bunch of different scenarios that they weren’t explicitly trained for, but they were never designed for precise dna analysis. It is their architecture that prevents them from other long horizon tasks like playing chess and the way that they represent text is why they can never count the letters in strawberry (most have this specific question hard-coded in their training data now).

Anyone who believes that LLMs are going to solve cancer either has no idea how they work or has been one-shotted from talking to Claudia

no wonder the staff voted with their feet

Rather than rebelling against google, it seems like they where just poached by anthropic a short while before the project shut down:

[Alphahold VP and nobel winner] Jumper announced in June that he was leaving DeepMind to join Anthropic. A couple of his colleagues jumped ship to Anthropic, as well. DeepMind has confirmed to the Times that it also moved staff members internally to Gemini-focused projects. Other former staff members were reassigned to Alphabet’s drug-discovery company Isomorphic Labs, which is a DeepMind spin-off.

Also apparently young leo is getting married to Dario’s Chief of staff this weekend lmaou

“From my perspective as an uninformed spectator, I would expect the bets made by Situational Awareness to be positive EV over the long run.”

Actually the value of his calls in the long run is 0, don’t worry I’m giving this hot financial tip for free. Do these dorks not understand options work? I guess neither did young leo.

Remember, there’s no crying in the casino. *though i have a feeling Leo will continue to fail upward ;_;

he didn’t hedge on long enough timelines

Duh. Everybody knows the market never stays irrational for long, and you should just tough it out.

The LW insider thinks Aschenbrenner is trying to save a stake in Anthropic. I think that is very likely because the press release stressed “public stock position” and Anthropic is privately owned.

Edit / I don’t think Thomas Kwa of METR and LW understands leverage. The idea is that if you borrow $40.5b on $4.5b of collateral, you have $45b to invest but a 10% drop will wipe you out. Before you get there the bank will knock on your door and say “margin call! either you provide more collateral or we seize your assets.”

By 2025/26, these machines will outpace many college graduates.

tick tick tick tick

(I am choosing to forego the obvious jibe that the models fulfilled this prophecy by dragging many college graduates down beneath them)

Here is the first sentence: "You can see the future first in San Francisco. "

I feel like I already know everything I need to know about him.

good grifter

Not denying that he isn’t also a grifter, but I bet he is a true believer and he had blindly bet his (and other people’s) money on “line goes up” exactly like his scenario said and that is why he is the first one to crack.

The reason that they are destroying the books is to avoid the accusation that the scanning constituted copying, an irony that we’ve discussed previously, on Awful while trying to understand which court cases are relevant. Anthropic actually hired the same guy, Tom Turvey, who designed Google Books’ ingestion process, so I’m thinking of this as a sequel to Google Books; hopefully the courts won’t take a decade this time.

Salt water is famously a forgiving environment to build in, and oceanic weather is benign and predictable. Not that it would be a problem anyway, because move-fast-and-break-things people can be trusted to do the right thing when it comes to running delicate equipment in places more hostile than mid california, and will not skimp on staffing and maintenance. You can look at all the the successful seasteading operations to see how well this will go.

There are also no problems I can foresee with putting a bucketload of fissiles in international waters, with no scope for finally uniting old-school piracy with new-school piracy, either. This plan is great, and no-one will have a problem with it.

but you see, in case of accident they can just dump reactor core into seawater which will provide ample cooling and shielding. no biggie, just put multi thousand ton object that requires a village of engineers to work out on high seas

there are normal nuclear powerplants that use seawater as coolant and they’re all on shore. these require a lot of corrosion resistant heat exchangers along the way. there is one power barge like this, it provides power to russian far east, whole 70MW of it, costed 414 million dollars and needs 300 people onsite. your guess is as good as mine how many of them are military. 70MW, aka two or three aeroderivative gas turbines

Oh, I’ve seen this one before. At least we know where the LLM who gave them this pitch scraped the idea from.

I once listened to a talk by a guy from german OSHA that hit a lot of the good points. Perceived productivity opposed to actual productivity from the METR studies. Mental load of alien code review. (This was a company-internal IT-expert exchange thingy.)

A question in Q&A (submitted anonymously on slido like a sniveling coward) was deadass “Have you ever coded with a frontier model?”

Funnily enough they said that back then too. What a weird coincidence!

I’d say it’s all of that, plus the fact that a lot of the cult stuff are also present in how organised rationalism works in general so you need to believe that such a tradeoff can be mostly worth it, otherwise it’s time to really get packing.

Although a more charitable read could be that the intended audience are people still under cult influence who need to be communicated to (in the most non-judgemental way possible so as to not trip any cult induced outgroup-o-phobia) that it’s entirely possible to feel you are in a situation with positive and productive elements that is also a fucking cult that is eating you alive.

The extended framing of how ‘high-control literature’ applies to the MAPLE situation probably also caters to those sensibilities, most other readers probably already got that it’s a cult as soon as stuff like dear leader halting his daily two-hour speech when someone almost nodded off to make everyone stand up and stare at them in silence for ninety minutes started being mentioned. Or the constant surveillance and brainwashing. Or keeping everyone within a hair’s width from collapse from exhaustion at all times. Or the isolation retreats where your only human contact is a few minutes per week of getting told off by dear leader. Or <gestures at everything>.

Having said all that, the positive aspects of MAPLE the author manages to come up with seem to amount to providing life structure and long term goals, which really doesn’t seem like much:

Again I’ll say: there was a great deal of good at MAPLE. While many people were being impacted in the ways above, many were also becoming less anxious, more grounded, more relationally attuned, less distracted and more focused, more capable of doing chores and taking on responsibility without internal resistance, physically healthier, deepening in their spiritual practice, and more. Most people experienced some amount of the positive and the negative, which is a large part of why many former residents say their relationship to MAPLE is “complicated.”

I don’t think you have to be hypercritical of that part. Sure, rose-colored glasses could play into it but he calls the bad stuff bad and doesn’t justify it via the good stuff.

Also the stuff he describes seems plausibly achievable through buddhist spiritial practice:

In many ways MAPLE delivered: life-long friendships, freedom from past addictions and limiting self-narratives, strengthened character, and, for many of us, profound spiritual openings and realizations.

This Soryu figure seems fairly well accredited in that department. He just also happens to be a megalomaniacal control freak.

(Also, why does he have to share a name with Asoka? I’m dying a little.)

edit: I knew I spelt megalomaniacal wrong…

Yeah, that is really depressing.

I think it’s a sunk cost thing for the authors, but also it could be fear of coming off as “disgruntled” to people they’re hoping to communicate with. They still have hope of changing these communities, but they know these communities are very hostile and dismissive towards any criticism, especially criticism from perceived “enemies”. They also don’t want to lose whatever power and respect they have in those communities. So emphasising how great these groups are and how smart all the members are and how much they got out of being a member is a tactic to try and get around all that hostility. They want to flatter the remaining members enough that they’ll retain their respect for them. Which in itself is a sad comment on the attitudes common in these communities.

Having said all that, the poor man is not long out of a cult and has taken a big step by publicly blowing the whistle. That takes some backbone.

yet another autodidact with a cushy job in the Effective Altruism grantland

How do I get one of these cushy jobs?

The actual worst man, and also just fucking sad. Imagine knowing that not only did you peak in high school, but that you peaked specifically in the moment when you fucked up and lost the state championship. Like, you can just smell the empty beer cans and tears piled up around this guy’s posts.

It was Yudkowsky-2008 who was astonished that “these CEOs and CTOs and hedge-fund traders, these folk of the mid-level power elite, seemed happier and more alive,” prompting David Gerard to ask “My dude, have you never heard of cocaine?” I don’t know if he had moved to the Bay Area yet but Yudkowsky-2025 had thoughts on LSD.

It gets worse. I found another article about the same podcast: Sam Altman says AI won’t shorten the workweek because humans secretly enjoy being busy.

“Technology, for a long time, has been promising people that they’re going to work less and they’re going to have all this leisure,” Altman said. “But somehow we never get the promise of the four-hour workweek at mass scale in society,” he added. “And I don’t expect AI to change that.”

“It’s like a relative game. People are very focused on how they’re doing relative to other people,” the CEO said.

“I think we’re all going to be much busier than we thought we were supposed to be in a post-superintelligence world. We’re still going to complain about it, but secretly we’re going to be happy,” Altman concluded.

Observe the power of a rhetorician who recycles singularitarian AND libertarian talking points simultaneously!

…damn, it’s not really very impressive. at all.

The chuds are all at r/accelerate now. “We’re the unapologetically Pro-AI alternative to r/singularity, r/futurology, r/artificial & r/technology which are becoming filled with Luddites, Techno-Decelerationists, Techno-Reactionaries, & Anti-AIs.”

And a follow-up by talia ringer, who observes that there have always been gaps between the type-theoretic underpinnings of things like the lean prover and their actual implementation, and this hasn’t been so much of an issue til now because theorem provers haven’t had the attention of people in high places, and the type-theoreticians have been able to catch up in due course.

https://mathstodon.xyz/@TaliaRinger/117005740997367321

My big worry right now is that if organizations continue to fund the crap out of Al for formal proof research (and to generally support implementation and maintenance of proof assistants like Lean as part of that effort) but don’t bother funding the type theory side of things, those gaps will grow larger and will be exploited more often by Al tools via reward hacking. Whereas people tend to only exploit kernel bugs to make a point that the bug exists. Thus proof assistants will grow less trustworthy over time.

Anyone want to place any bets on whether or nor the big llm companies are going to fund academic research that isn’t obviously mechanisable right now and won’t yield any clickbait headlines?

they’re all getting stiffies from the idea of black market models