The AI people are still infatuated with math. The Epoch AI staff, after being thoroughly embarrassed last year by the FrontierMath scandal, have now decided to make a new FrontierMath Open Problems benchmark, this time with problems that people might give a shit about!

I decided to look at one of the easiest “moderately interesting” problems and noticed that GPT-5.2 Pro managed to solve a warm up version of the problem, i.e. a version that had been previously solved. Wow, these reasoning models sure are capable of math! So I was curious and looked at the reasoning trace and it turns out that … the model just found an obscure website with the right answer and downloaded it. Well, I guess you could say it has some impressive reasoning as it figures out how to download and parse the data, maybe.

Hey, you’re selling them short: there are also ReLU and softmax activation functions thrown around here and there. Clankers aren’t just linear transformations! /j

I am a computer science PhD so I can give some opinion on exactly what is being solved.

First of all, the problem is very contrived. I cannot think of what the motivation or significance of this problem is, and Knuth literally says that it is a planned homework exercise. It’s not a problem that many people have thought about before.

Second, I think this problem is easy (by research standards). The problem is of the form: “Within this object X of size m, find any example of Y.” The problem is very limited (the only thing that varies is how large m is), and you only need to find one example of Y for each m, even if there are many such examples. In fact, Filip found that for small values of m, there were tons of examples for Y. In this scenario, my strategy would be “random bullshit go”: there are likely so many ways to solve the problem that a good idea is literally just trying stuff and seeing what sticks. Knuth did say the problem was open for several weeks, but:

1. Several weeks is a very short time in research.
2. Only he and a couple friends knew about the problem. It was not some major problem many people were thinking about.
3. It’s very unlikely that Knuth was continuously thinking about the problem during those weeks. He most likely had other things to do.
4. Even if he was thinking about it the whole time, he could have gotten stuck in a rut. It happens to everyone, no matter how much red site/orange site users worship him for being ultra-smart.

I guess “random bullshit go” is served well by a random bullshit machine, but you still need an expert who actually understands the problem to read the tea leaves and evaluate if you got something useful. Knuth’s narrative is not very transparent about how much Filip handheld for the AI as well.

I think the main danger of this (putting aside the severe societal costs of AI) is not that doing this is faster or slower than just thinking through the problem yourself. It’s that relying on AI atrophies your ability to think, and eventually even your ability to guard against the AI bullshitting you. The only way to retain a deep understanding is to constantly be in the weeds thinking things through. We’ve seen this story play out in software before.

I was pissed when my (non-academic) friends saw this and immediately started talking about how mathematicians and computer scientists need to use AI from now on.

scott jumpscare

Baldur Bjarnason’s essay remains evergreen.

Consider homeopathy. You might hear a friend talk about “water memory”, citing all sorts of scientific-sounding evidence. So, the next time you have a cold you try it.

And you feel better. It even feels like you got better faster, although you can’t prove it because you generally don’t document these things down to the hour.

“Maybe there is something to it.”

Something seemingly working is not evidence of it working.

• Were you doing something else at the time which might have helped your body fight the cold?

• Would your recovery have been any different had you not taken the homeopathic “remedy”?

• Did your choosing of homeopathy over established medicine expose you to risks you weren’t aware of?

Even when looking at Knuth’s account of what happened, you can already tell that the AI is receiving far more credit than what it actually did. There is something about a nondeterministic slot machine that makes it feel far more miraculous when it succeeds, while reliable tools that always do their job are boring and stupid. The downsides of the slot machine never register in comparison to the rewards. Does it feel so miraculous when I get an idea after experimenting in Mathematica?

I feel like math research is particularly susceptible to this, because it is the default that almost all of one’s attempts do not succeed. So what if most of the AI’s attempts do not succeed? But if it is to be evaluated as a tool, we have to check if the benefits outweigh the costs. Did it give me more productive ideas, or did it actually waste more of my time leading me down blind alleys? More importantly, is the cognitive decline caused by relying on slot machines going to destroy my progress in the long term? I don’t think anyone is going to do proper experiments for this in math research, but we have already seen this story play out in software. So many people were impressed by superficial performances, and now we are seeing the dumpster fire of bloat, bugs, and security holes. No, I don’t think I want that.

And then there is the narrative of not evaluating AI as an objective tool based on what it can actually do, but instead as a tidal wave of Unending Progress that will one day sweep away those elitists with actual skills. Random lemmas today mean the Millennium Prize problems tomorrow! This is where the AI hype comes from, and why people avoid, say, comparing AI with Mathematica. To them I say good luck. We have dumped hundreds of billions of dollars into this, and there are only so many more hundreds of billions of dollars left. Were these small positive results (and significant negatives) worth hundreds of billions of dollars, or perhaps were there better things that these resources could have been used for?

Don’t worry, there’s always Effective Altruism if you ever feel guilty about causing the suffering of regular people. Just say you’re going to donate your money at some point eventually in the future. There you go, 40 trillion hypothetical lives saved!

This somehow makes things even funnier. If he had any understanding of modern math, he would know that representing a set of things as points in some geometric space is one of the most common techniques in math. (A basic example: a pair of numbers can be represented by a point in 2D space.) Also, a manifold is an extremely broad geometric concept: knowing that two things are manifolds does not meant that they are the same or even remotely similar, without checking the details. There are tons of things you can model as a manifold if you try hard enough.

From what I see, Scoot read a paper modeling LLM inference with manifolds and thought “wow, cool!” Then he fished for neuroscience papers until he found one that modeled neurons using manifolds. Both of the papers have blah blah blah something something manifolds so there must be a deep connection!

(Maybe there is a deep connection! But the burden of proof is on him, and he needs to do a little more work than noticing that both papers use the word manifold.)

Kolmogorov complexity:

So we should see some proper definitions and basic results on the Kolmogorov complexity, like in modern papers, right? We should at least see a Kt or a pKt thrown in there, right?

Understanding IS compression — extracting structure from data. Optimal compression is uncomputable. Understanding is therefore always provisional, always improvable, never verifiably complete. This kills “stochastic parrot” from a second independent direction: if LLMs were memorizing rather than understanding, they could not generalize to inputs not in their training data. But they do. Generalization to novel input IS compression — extracting structure, not regurgitating sequences.

Fuck!

Nonsensical analogies are always improved by adding a chart with colorful boxes and arrows going between them. Of course, the burden of proof is on you, dear reader, to explain why the analogy doesn’t make sense, not on the author to provide more justification than waving his hands really really hard.

Many of these analogies are bad as, I don’t know, “Denmark and North Korea are the same because they both have governments” or something. Humans and LLMs both produce sequences of words, where the next word depends in some way on the previous words, so they are basically the same (and you can call this “predicting” the next word as a rhetorical flourish). Yeah, what a revolutionary concept, knowing that both humans and LLMs follow the laws of time and causality. And as we know, evolution “optimizes” for reproduction, and that’s why there are only bacteria around (they can reproduce every 20 minutes). He has to be careful, these types of dumbass “optimization” interpretations of evolution that arose in the late 1800s led to horrible ideas about race science … wait a minute …

He isn’t even trying with the yellow and orange boxes. What the fuck do “high-D toroidal attractor manifolds” and “6D helical manifolds” have to do with anything? Why are they there? And he really thinks he can get away with nobody closely reading his charts, with the “(???, nothing)” business. Maybe I should throw in that box in my publications and see how that goes.

I feel like his arguments rely on the Barnum effect. He makes statements like “humans and LLMs predict the next word” and “evolution optimizes for reproduction” that are so vague that they can be assigned whatever meaning he wants. Because of this, you can’t easily dispel them (he just comes up with some different interpretation), and he can use them as carte blanche to justify whatever he wants.

Maybe I should apply to be a director of AI safety at Meta. I know one safety measure that works: don’t use AI.

I guess I can check back in six months to see how they’re doing … wait a minute, they were saying the same things six months ago, weren’t they? That’s a bummer.

$1000 a week?? Even putting aside literally all of the other issues of AI, it is quite damning that AI cannot even beat humans on cost. AI somehow manages to screw up the one undeniable advantage of software. How do these people delude themselves into thinking that the dogshit they’re eating is good?

As a sidenote, I think after the bubble collapses, the people who predict that there will still be some uses for genAI are mostly wrong. In large part, this is because they do not realize just how ruinously expensive it is to run these models, let alone scrape data and train them. Right now, these costs are being subsidized by venture capitalists putting their money into a furnace.

I admire how persistent the AI folks are at failing to do the same thing over and over again, but each time coming up with an even more stupid name. Vibe coding? Gas Town? Clawdbot, I mean Moltbook, I mean OpenClaw? It’s probably gonna be something different tomorrow, isn’t it?

It’s a big club and you ain’t in it!

Holy shit, I didn’t even read that part while skimming the later parts of that post. I am going to need formal mathematical definitions for “entangled limit”, “all possible computations”, “everything machine”, “maximally nondeterministic”, and “eye wash” because I really need to wash out my eyes. Coming up with technical jargon that isn’t even properly defined is a major sign of math crankery. It’s one thing to have high abstractions, but it is something else to say fancy words for the sake of making your prose sound more profound.

I study complexity theory so this is precisely my wheelhouse. I confess I did not read most of it in detail, because it does spend a ton of space working through tedious examples. This is a huge red flag for math (theoretical computer science is basically a branch of math), because if you truly have a result or idea, you need a precise statement and a mathematical proof. If you’re muddling through examples, that generally means you either don’t know what your precise statement is or you don’t have a proof. I’d say not having a precise statement is much worse, and that is what is happening here.

Wolfram here believes that he can make big progress on stuff like P vs NP by literally just going through all the Turing machines and seeing what they do. It’s the equivalent of someone saying, “Hey, I have some ideas about the Collatz conjecture! I worked out all the numbers from 1 to 30 and they all worked.” This analogy is still too generous; integers are much easier to work with than Turing machines. After all, not all Turing machines halt, and there is literally no way to decide which ones do. Even the ones that halt can take an absurd amount of time to halt (and again, how much time is literally impossible to decide). Wolfram does reference the halting problem on occasion, but quickly waves it away by saying, “in lots of particular cases … it may be easy enough to tell what’s going to happen.” That is not reassuring.

I am also doubtful that he fully understands what P and NP really are. Complexity classes like P and NP are ultimately about problems, like “find me a solution to this set of linear equations” or “figure out how to pack these boxes in a bin.” (The second one is much harder.) Only then do you consider which problems can be solved efficiently by Turing machines. Wolfram focuses on the complexity of Turing machines, but P vs NP is about the complexity of problems. We don’t care about the “arbitrary Turing machines ‘in the wild’” that have absurd runtimes, because, again, we only care about the machines that solve the problems we want to solve.

Also, for a machine to solve problems, it needs to take input. After all, a linear equation solving machine should work no matter what linear equations I give it. To have some understanding of even a single machine, Wolfram would need to analyze the behavior of the machine on all (infinitely many) inputs. He doesn’t even seem to grasp the concept that a machine needs to take input; none of his examples even consider that.

Finally, here are some quibbles about some of the strange terminology he uses. He talks about “ruliology” as some kind of field of science or math, and it seems to mean the study of how systems evolve under simple rules or something. Any field of study can be summarized in this kind of way, but in the end, a field of study needs to have theories in the scientific sense or theorems in the mathematical sense, not just observations. He also talks about “computational irreducibility”, which is apparently the concept of thinking about what is the smallest Turing machine that computes a function. This doesn’t really help him with his project, but not only that, there is a legitimate subfield of complexity theory called meta-complexity that is productively investigating this idea!

If I considered this in the context of solving P vs NP, I would not disagree if someone called this crank work. I think Wolfram greatly overestimates the effectiveness of just working through a bunch of examples in comparison to having a deeper understanding of the theory. (I could make a joke about LLMs here, but I digress.)

Surely this is a suitable reference for a math article!

I think Aumann’s theorem is even narrower than that, after reading the Wikipedia article. The theorem doesn’t even reference “reasoning”, unless you count observing that a certain event happened as reasoning.

I’d say even the part where the article tries to formally state the theorem is not written well. Even then, it’s very clear how narrow the formal statement is. You can say that two agents agree on any statement that is common knowledge, but you have to be careful on exactly how you’re defining “agent”, “statement”, and “common knowledge”. If I actually wanted to prove a point with Aumann’s agreement theorem, I’d have to make sure my scenario fits in the mathematical framework. What is my state space? What are the events partitioning the state space that form an agent? Etc.

The rats never seem to do the legwork that’s necessary to apply a mathematical theorem. I doubt most of them even understand the formal statement of Aumann’s theorem. Yud is all about “shut up and multiply,” but has anyone ever see him apply Bayes’s theorem and multiply two actual probabilities? All they seem to do is pull numbers out of their ass and fit superexponential curves to 6 data points because the superintelligent AI is definitely coming in 2027.