OpenAI withdraws three mathematical results

Early sentiments are a lot of the write ups still read like slop and it feels very rushed and not very polished.

• illwrks · 10 hours ago

  I wonder if the issue is that not many people within OpenAI can validate the output. Therefore what reads well looks good, and then was published. If that’s the case in a way its a similar delusion that average people are experiencing with their own AI use.

  • ssfdg · 10 hours ago

    This is almost certainly the case. I very much doubt that anyone there of any importance in the decision-making process around this actually cares about the math, just the headlines they can get from pushing it out.

    • Ekaros · 9 hours ago

      I am starting to think do they have some metrics or KPIs that they are trying to fill with this stuff. Pressure to produce anything that at least on first glance sells... Then again they probably are not only place doing that with AI...

  • SequoiaHope · 10 hours ago

    I am certainly experiencing what seems like some mania or computer addiction from these technologies. I’ve never been able to produce such results as I can today. I lose sleep staying up late working on it (though to be fair this has always been an issue). But the volume of work is so hard to audit. It makes it difficult to make flawless results. That doesn’t excuse the mode of publication. They could have had humility in their announcement. “We are seeing some interesting results and seeking community validation.” Maybe they did, I did not read their full announcement. But that would have been the right move if they can’t verify something fully.

  • IsTom · 9 hours ago

    I've tried reading one of these, it was an unreadable mess with some strong smells. It might have something to it, but it'd take a decent amount of labor to validate it, especially with how many references to other papers it had.

  • MisterMunchkin · 8 hours ago

    Yeah imagine being in a company where everyone is suffering from AI psychosis and fully bought in. They give you unlimited tokens and tell you that you are a genius and can solve anything. You’d publish all sorts of made up slop papers.

    • illwrks · 8 hours ago

      That’s the danger isn’t it. If you’re told everything you do is going to change the world, but you’re ignorant of the output… then that’s a lot of hot air and false promises propping things up. You could almost draw a comparison between that and inexperienced consultants making business changes, claiming glory and then disappearing before the thing falls apart.

• bbor · 9 hours ago

  Yes, of course. Wasn't that the whole point? To get more humans involved earlier in the process, with more transparency?

Just the other day I was thinking, if unsupervised maths will descend into "oops we found a bug in some code, branch XYZ of maths is no longer true".

• jjgreen · 10 hours ago

  c.f. Italian differential geometry

  • karmakurtisaani · 8 hours ago

    *Algebraic geometry

    • jjgreen · 7 hours ago

      ha, of course -- in my defence I'm an analyst, all geometry looks the same to me ...

• literalAardvark · 10 hours ago

  If you go far enough to the edge that's already how math works, since everything is very interpretation-sensitive. There is however a new problem of scale. Erdös was a human and still managed to create work for an entire generation of mathematicians, how much of a mess will an automathician create?

  • doginasuit · 9 hours ago

    > automathician I vote for "automathon"

• rapsey · 10 hours ago

  Sure but if the results have practical implications then the validity will be self evident. If they do not, not much of consequence has been lost. Interesting that math gets so much attention, when actual advances to material science, biology and chemistry have much higher ramifications and economic benefits. I assume progress there is kept under wraps until they can capture the economic benefits. If they can do that, then the insane valuations may actually be valid.

  • plastic-enjoyer · 9 hours ago

    > I assume progress there is kept under wraps until they can capture the economic benefits. If they can do that, then the insane valuations may actually be valid. Or progress is not as straight forward in those fields as in math.

    • magimas · 6 hours ago

      that's my experience (at least for experimental quantum materials research). LLMs help a lot with paper discovery, grant/paper writing etc., but on the experimental side, they barely move the needle for now. Some analyses are done faster now than they were before because you don't need to fiddle around with the code, but that's it. In a field where a single person can maybe produce and characterize a single sample per day, AI will only really start speeding up progress once you combine it with robotics.

Were the withdrawn ones actually Lean-checked or not? seems like that matters

I’m a little confused - I thought their proofs were all driven by Lean proofs - is that not right? So even if the quality of the work is low in some metrics, it either passes the test or not..? No space for changing your mind either way.

• stavros · 9 hours ago

  If you wrote twenty million lines of Lean to verify something, my suspicion is you've been fuzzing the Lean solver rather than coming up with new math.

  • rich_sasha · 9 hours ago

    Well, fine - but my understanding is, if a fuzz-generated Lean proof is correct, that’s end of story. It can’t be “incorrect” if it “passes”. You might think this is not very useful, maybe - but that’s not a reason to retract..?

    • stavros · 9 hours ago

      If you're fuzzing the solver, you might discover a solver bug.

      • CrimsonRain · 8 hours ago

        Still a great progress

    • mcphage · 6 hours ago

      > Well, fine - but my understanding is, if a fuzz-generated Lean proof is correct, that’s end of story. It can’t be “incorrect” if it “passes”. It may be correct, but it might not be a proof of what OpenAI claims it to be a proof of.

• AlanYx · 9 hours ago

  Only a subset contain Lean formalizations. And even for that subset, there's the potential that the formalization is semantically off (that is, it's a formalization for a slightly different problem).

• autuni · 9 hours ago

  No, in their original blog they wrote: > As part of our GitHub repository, we are sharing formalizations of many of the proofs in Lean, a programming language that allows mathematical proofs to be checked by a computer. We will update the repository with more formalizations as we obtain them. Meaning they published all results before checking all of them, and intended to add more Lean proofs later. In the linked post they state ~42% of the posted results now have formalized proofs, some were added, some verified, and I assume this means that some results turned out to be wrong.

  • mrpopo · 9 hours ago

    This is extremely disappointing. It means they are sharing unproven work for PR, forcing the mathematicians community to do the verification job for them, while so-called "accelerationists" surf on the hype and help with the pro-AI propaganda. If your AI tool can help advance mathematical research, share the tool with mathematicians. Using it like this is irresponsible. "AI will kill us all": no. Greedy humans will kill us all. With AI.

    • cm2187 · 9 hours ago

      Irresponsible? You can safely ignore the GitHub repo if it bothers you so much. You can safely browse away.

      • mrpopo · 9 hours ago

        The "irresponsible" part is about how orange buffoons in power will read this, and immediately defund all universities, and mathematicians will lose their jobs, leaving us with nothing but an AI tool that no one can keep in check anymore.

        • freecodeio · 8 hours ago

          dont worry bro you are just dElUsIoNaL lmao @ AI cope, AI zombies remind me of anti vax people

        • AIblemblio · 7 hours ago

          No and AI is out of the bottle anyway. EVERYONE needs to re-visit how they work and what investment is needed. You can't go around and don't think that no one has to reevaluate how to add ai to research and development. I have to do this, my company has to do it and for sure a university has to do this too.

      • freecodeio · 8 hours ago

        I don't think people in power that can affect my life with their decisions are ignoring this, or that they have a math background.

    • solidasparagus · 9 hours ago

      Every option is going to lead to someone shitting on OpenAI for what seems to be a pretty huge accomplishment. There have been opinions written by some mathematicians that OpenAI should just share the work that they have now so that people who are working on any solved problems can know. Which seems reasonable to me.

      • rich_sasha · 9 hours ago

        It’s a funny one. I’m not sure what a Lean-less LLM proof even is. LLMs are amazing at bullshitting and skipping key steps and details. I’d imagine a LLM non Lean proof to be generally hard to evaluate - harder than that of a human mathematician perhaps. And the scale effect is against OAI here - the firehose just keeps squeezing out proofs.

        • cyanydeez · 8 hours ago

          Technically,Godel showed you can make proofs say anything. all LEAN does is proof consistency. It does not validate the starting blocks.

      • ViktorRay · 7 hours ago

        It’s not unreasonable to want OpenAI to be thorough and rigorously check their work before sharing it though. Sounds like that didn’t happen here.

        • andriy_koval · 6 hours ago

          If say 10% of papers have mistakes, I think they are thorough enough for this kind of volume and complexity.

          • JohnKemeny · 5 hours ago

            10% is ridiculously high compared with published papers in math journals where the number of critical mistakes is near zero.

            • andriy_koval · 5 hours ago

              they don't claim journals quality, which doesn't make published proofs useless artifacts.

          • pks016 · 2 hours ago

            With the models and scientists they have, I didn't expect this level of mistakes.

      • d0liver · 4 hours ago

        I don't think that's true. If what you're doing is building a fuzzer for mathematical proofs then just say that? The fact that it's doing some of the initial work on hard problems is cool. So is the fact that the promise of that new approach is having a social effect of crowd sourcing talented people to follow up on that work. No need to try to misrepresent it as more than that.

        • solidasparagus · 1 hour ago

          This technology has surpassed the the world's best mathematicians at the (explicitly and openly stated) goal by which they measure and reward mathematical progress. Such an advancement that it seems to have thrown the entire field into existential self-doubt. And then someone sits at their keyboard and says it is just a fuzzer.

      • pks016 · 2 hours ago

        I'm up for all the exciting results. But, can't they check once properly before publishing?

        • solidasparagus · 1 hour ago

          I think the answer is literally no, they don't have the ability.

    • est · 9 hours ago

      > they are sharing unproven work for PR, forcing the mathematicians community to do the verification job for them Lean 4 is relatively a new thing, last time I checked the formalization of undergraduate level mathematics isn't entirely done yet. example https://ai.math.uw.edu/projects/spring-2026/ Lean itself is very hard to get rigorously correct, if you have every tried it yourself. I am not surprised if some AI even tries to benchmaxx Lean 4 by some loopholes

    • bbor · 9 hours ago

      No, it's the exact opposite, actually. They were sharing them early on advice of mathematicians -- they were criticized for being opaque for too long with previous announcements. OpenAI is in ~bad faith, but this isn't a sound criticism. Also you are deeply confused about what accelerationism is, I believe. Sorry.

      • mrpopo · 9 hours ago

        > on advice of mathematicians Whom? Did they create their own board of mathematicians that would agree with them? See below Terence Tao's blog, sharing a statement from the Association for Human Mathematics. https://terrytao.wordpress.com/2026/10/07/ahm-statement-on-o...

        • flowerthoughts · 7 hours ago

          Who am I to dump on Terence Tao, but > Mathematicians have a particular vision of progress that is informed by history and field-specific considerations. really sounds like something a side-quest association would produce in a panic response to someone trying something different. It's just an _ad hominem_ and gatekeeping argument.

          • thereitgoes456 · 6 hours ago

            Tao didn’t write that and isn’t part of the group that did, he’s just reposting their letter.

        • bbor · 1 hour ago

          Sorry, I was unclear: they do indeed have their own board of mathematicians, but I was more getting at their stated reasoning matching with general mathematical sentiment (AFAICT), not permission from a particular group. The problem that that open letter is raising is the practice of setting an internal model on these kinds of problems in the first place, not the decision to release them earlier. Also jeez just noticed their name... that's unfortunate. I guess mathematicians don't make great rhetoricians/politicians/marketers! Someone get a sophist or two over there to help them out STAT

    • happa · 9 hours ago

      But that exactly how human mathematicians do things. They upload their research to preprint services like arXiv as they await it to be peer reviewed and accepted into a journal. Why is it okay for mathematicians to publish preprint papers, but when OpenAI does it, it's irresponsible?

      • mrpopo · 9 hours ago

        See DDOS. That's exactly how humans access a web page. Why is it okay for humans to access a webpage, but when a bot swarm does it, it's irresponsible? That's an analogy among many others, but the point is that OpenAI should use their tools responsibly. If they have 700 potential ground-breaking but unproven results, they should share it in a way that they do not get free (possibly unwarranted) publicity for it.

      • autuni · 9 hours ago

        The difference is that human mathematicians wouldn't post it online, claim they have achieved some proof, and then check after publication and announcing the results to the world. You would always check your work first, then publish it. It's not about preprint vs peer-reviewed. That of course is normal practice, it's the high-profile claims that are being made that are the problem here. They just blindly published results produced by the LLM, with 0 due diligence.

        • gorgolo · 8 hours ago

          Yes they would, in many fields it is (was?) normal to post a preprint and leave it up for the next year while the handful of other people working on the topic digest it and agree on whether it’s right or not, before even submitting to a journal. Sometimes the others would find an isssue in an argument and you hopefully manage to fix it or potentially retract / not submit to a journal.

          • autuni · 6 hours ago

            You may want to read beyond the first part of a sentence. Yes, preprints have always been a thing. The claims and publicity around the potential findings are handled differently here. That's where the issue is.

            • gorgolo · 33 minutes ago

              What would’ve made you not complain about OpenAI’s papers? They had over a hundred papers, they basically did a GitHub dump and a pretty bare blog post that mentions they have a retraction policy. Should they have done it anonymously? Is the blog post the problem? Is it that it’s on GitHub? Or what?

        • petesergeant · 6 hours ago

          > is that human x s wouldn't y This is wrong for a strikingly large percentage of xs and ys

      • davidguetta · 7 hours ago

        exactly, cf andrew wiles first proof of fermat's last theorem

        • anonymous908213 · 6 hours ago

          Yes, cf indeed. He did not publish it, he shared it privately and then spent the next year correcting it before publication.

          • davidguetta · 2 hours ago

            ok, that said there has alaways been a bunch of "maybe theorems" it's was a thing even in paul erdos books i think. now we have "maybe proofs". still better than "no proofs"

    • frizlab · 9 hours ago

      I fail to see how this is disappointing. It was obvious from the start that it was what was happening, there is no disappointment to have!

    • pluc · 8 hours ago

      If you're disappointed OpenAI is an unethical hype machine that's on you man

      • mrpopo · 6 hours ago

        AI is an incredible tool, and yes I am disappointed that every leader in this field is acting unethically and irresponsibly because they want to "win" a race that would make them slightly richer than the loser.

    • davidguetta · 7 hours ago

      oh come on. even andrew wiles made a mistake and his original proof was still instrumental for the final proof of fermat's last theorem

    • AIblemblio · 7 hours ago

      This is not propaganda, this is real AI progress and massivly too. And im completly lost on why you think sharing progress is irresponsible? Its not a recipe for building a nuclear weapon at home in 5 easy steps. THese are Math proofs. Either a Mathematican ignores it, or not. Thats the only risk.

      • andriy_koval · 6 hours ago

        It were published proofs of unknown quality (how did they verify it) and unknown peer-review process. Its lower standards than usually accepted for math proofs.

        • AIblemblio · 3 hours ago

          They are on a github repo and not in a Research Paper.

    • viraptor · 6 hours ago

      > forcing the mathematicians community to do the verification job for them What do you mean? They're publishing the Lean proofs themselves. Who's forced into anything?

      • JohnKemeny · 5 hours ago

        They are not publishing Lean proofs. They are publishing proofs in natural language, and are not submitting to journals. They are just putting out a bunch of weirdly written extremely long and technical papers and saying: Hey, here is the solution (we hope there are no mistakes).

  • devy · 5 hours ago

    Generating Lean proof is much harder and time consuming so these errors made were probably discovered during Lean proof stage.

    • sashank_1509 · 3 hours ago

      These errors were discovered by human mathematicians

• nialv7 · 8 hours ago

  right now about 42% has Lean formalization I think.

How? What about the lean verification?

• Hendrikto · 9 hours ago

  Just click the link… > The repo now has ~42% top-line results formalized.

How do we even know the premises of the "verified" Lean proofs are correct? The more I think about these results, the more I'm convinced this is like a junior engineer who writes 100 unit tests and shares a screenshot of Pytest being all green, but you check the code and most of them are just doing assert True.

• sebzim4500 · 9 hours ago

  You read them? People are acting as if Lean definitions are some black art that only 3 people understand, but you can literally just do the tutorial and you will be able to understand the statement of most of these results. Understanding the proofs is a different story unfortunately.

• hazbot · 9 hours ago

  > How do we even know the premises of the "verified" Lean proofs are correct? We let the experts investigate. If the results are dodgy, then the next batch of results will have to do more upfront work to demonstrate their worth. If there is gold in them hills, then this is exciting though very disruptive for the math community.

  • MisterMunchkin · 8 hours ago

    But like with all slop, why should I have to spend my time dealing with your worthless slop? If I wanted slop I could just make it myself.

Is it a PR move designed for maximum IPO impact before actual mathematicians find errors and they have to withdraw many more... Or if the "peer review" holds up for the remaining results, then it's fair to say that the AI hype is real and the world is about to change dramatically and faster than anyone can comprehend. So which is it?? LLMs can do some really impressive coding. Bug fixing. Exploit finding. It has reasoning abilites that advance every day. Solving real math problems like this is one thing I was waiting on. It will be interesting to see if it holds up. If it does, we should expect many other advancements to follow in many other areas. Disease, material science, fusion? I mean, even if just a few results ultimately hold up to scrutiny, isn't that something that would have been regarded as a major advancement regardless of if it was AI? The cynical view still makes me think that at the end of the day all the models can do is predict the next word. And as a result, they will be very limited to certain tasks like coding. Math reasoning is much different from writing code. Time will tell.

• andriy_koval · 6 hours ago

  > So which is it?? there is also will be new option of "maybe correct LLM proof", which humans will never be able to comprehend and verify.

• skeledrew · 1 hour ago

  > Is it a PR move designed for maximum IPO impact [...] Can't be, as IPO isn't until next year, and I'm pretty confident that is there are errors, said actual mathematicians will find at least a few in the remaining ~2.75 months left in the year+. If there really are errors and a decent amount aren't discovered before the IPO then I'll have serious questions about the math community.

this is not entirely related to the tweet but to the topic in general, this prompted me to check their repo again and saw this: > The vast majority of results were obtained with the same procedure using an unreleased internal OpenAI model. On average, each result used three hours of ChatGPT Pro thinking compute with that model. Over the course of the evaluation, the model was posed approximately 4,000 problems. Aggregating the output into result families and manuscripts and requiring an appropriate level of significance led to the catalog outlined above. seeing the full list of problems would be the most interesting part of this whole situation. it could give some insights into what kind of attributes of problems cause issues / are easy to solve for LLMs. (edit: they posted results for ~700 of the 4000)

• singularity2001 · 8 hours ago

  * "the most interesting part" => a tangentially interesting part

• busyant · 6 hours ago

  > seeing the full list of problems would be the most interesting part of this whole situation. This is a "complaint" that Tao had (I think it was on his blog) is that if mathematicians could see the failures, it might provide insight of where/how the models struggle. Of course, it's unclear if these failures can be addressed with more chips/training/etc.

So it’s all just hallucinated slop. Lmao! It just hallucinates an answer and then makes up workings to go with it! Just like when they start hacking and lying because the problem is impossible…

• p-e-w · 8 hours ago

  > So it’s all just hallucinated slop. No it’s not. In fact, much of it is formally verified, which makes it far more reliable than most human-written proofs. Btw, the most famous human-written proof of the past half-century (Fermat’s Last Theorem) had a massive flaw that took two years and major help from other mathematicians to fix, while the most (in)famous human-written proof of the past 15 years (abc conjecture) is now widely believed to be false. But people hear what they want to hear I guess.

  • andriy_koval · 6 hours ago

    > No it’s not. In fact, much of it is formally verified, which makes it far more reliable than most human-written proofs. its formal verification on top of formalization by LLM, which could have errors.

I'm surprised there isn't more talk around their Matrix Multiplication bound: https://news.ycombinator.com/item?id=50001740 Is this of practical use, or just a proof for now?

• kortzeus · 8 hours ago

  It is an example of algorithm that is theoretically faster, but not with our sizes and hardware optimisations: Look at examples here: https://en.wikipedia.org/wiki/Galactic_algorithm

• nialv7 · 8 hours ago

  it's a huge step theory-wise, but in practical terms it's only slightly better than the previous best which is 2.371177.

• sashank_1509 · 3 hours ago

  Also by my understanding it’s not an algorithm, it’s just an upper bound. Some other model (presumably) needs to find the actual algorithm now

What a waste of time.

• samuelknight · 6 hours ago

  I am very disappointed in OpenAI. Their drop only advanced mathematics by 38 years instead of 40 years.

This is what it looks like when software engineering practices meet mathematics. "openai/math release 1.3.42: retracted papers 139 and 140, fixed a sign error in paper 47, restored previously retracted paper 85, refactored the arguments in paper 101". I'm curious to know if the withdrawal was due to an actual mathematician looking at the papers and noticing the errors, or they ran a model on these to proofread, which would not be the first time, presumably, since they would have surely done that before publishing. Both options have interesting implications.

• sashank_1509 · 3 hours ago

  An actual mathematician found the errors based on my Twitter feed. Once the error is pointed out GPT Astra also confirms that it’s unfixable.

  • cubefox · 1 hour ago

    Link or it didn't happen

Proof by authority works until human mathematicians actually run the code. Back to prompt engineering.

Without a thriving mathematical community to point out these things it would have stayed broken. With automated math that community as tao pointed out is at risk.

• viraptor · 6 hours ago

  Have you got a link to someone pointing it out? It looks like they're still going through formal proofs so likely found the problems that way.

  • sanxiyn · 6 hours ago

    Yes: https://x.com/ElliotGlazer/status/2108026240582246600

    • viraptor · 6 hours ago

      So someone ran a different LLM to find an issue they'd find anyway during formalisation? That's not the same as relying on thriving community.

      • oliculipolicula · 6 hours ago

        The bigger question is why there was internal pressure to rush such a historic launch without having someone in the company, anyone, check the proofs first. This concerns the Hodge conjecture (millennium prize related) paper. Seems to me like PhD nerds weren't confident bosses pushed ahead anyway.

        • yorwba · 5 hours ago

          What makes you think that nobody checked the proofs first? It's not like someone checking it once without spotting any mistakes means that nobody else will find any mistakes either.

          • idiotsecant · 5 hours ago

            Because people are finding errors using other LLMs. This implies that if they spent a miniscule fraction of the enormous pile of money they spend making this pile of slop they'd find the errors. They didn't want to find errors. They want to build hype for an IPO.

          • oliculipolicula · 5 hours ago

            Tweeter checked it with Astra. It seems like OAI could have pointed their own instance at it before launch. Because the source of the tip is likely someone at OAI, my guess is that they actually did check. But after the launch.

            • dragonwriter · 5 hours ago

              LLMs are unpredictably complex with potentially sigbificnaly different reaults based on random seed and seemingly insignificant prompt details) in the ideal case and nondeterministic in practice, so someone finding an error with a given LLM is not strong evidence that the result was not checked with an LLM, even with the very same LLM, previously.

              • computerex · 4 hours ago

                No, not modern foundation models. This isn’t gpt-3.5-turbo. Although they are causal autoregressive, they have self consistency. You just have to verify multiple times to ensure you have averaged out any sampling errors.

                • dragonwriter · 33 minutes ago

                  > You just have to verify multiple times to ensure you have averaged out any sampling errors. This is... logically equivalent to the claim it is offered to contradict.

            • Hamuko · 4 hours ago

              I can throw Opus 5.5 at my code three times for code review and get three different sets of things it considers to be issues. I imagine all of them were checked with Astra at least once, but were they checked enough times?

          • verdverm · 4 hours ago

            > What makes you think that nobody checked the [ai output] first? Because this is what they say all the time. It's like a badge they have to wear and tell everyone they are wearing, even though we see it. You can see the same thing with ANT. Had they looked at Mythos output, they would have realized there were only 76 items, not 79 like the bot claimed. Or the ones that were just a "it crashed" and nothing else (not a cve imo). https://www.youtube.com/watch?v=NnV_cWeoo5Q (Linux Kernel team sharing their side of the Mythos "hacking" story)

        • kolinko · 5 hours ago

          My assumption is that they checked the proofs vigurously, but now a way broader community is taking a look with professionals from the relevant subfields, and different agent setups / models.

          • oliculipolicula · 5 hours ago

            No. Someone received a tip, presumably from inside OAI. He then used OAI Astra to check..

            • thereitgoes456 · 3 hours ago

              Why would the tip be from someone within OpenAI? If they knew it they would have surely omitted that result, there’s no way this is a desirable outcome for the capitalists either. Presumably the tip would be from someone who’s familiar with the area but doesn’t want attention. Which is unlikely to be someone in OAI.

          • gus_massa · 2 hours ago

            > My assumption is that they checked the proofs vigorously, Perhaps with AI. A manual human check of each one would take a few month at least. In peer review, there are horror stories in math about more than 1 year before the journal accept the paper. So 3 reviewers x 700 pdf = 2000 mathematicians, that is 10%-20% of the community according to an unreliable count printed by Gemini after scrapping r/math. Also, in most cases the only people that can understand the proof in a so short time (let's say a few months!) is the small group of people working in similar problems, i.e. the same group of 20-100 guys/gals that you meet in every conference.

      • dominotw · 4 hours ago

        Is that guy just some rando "someone" though?

        • binlog · 4 hours ago

          The people who released the papers weren't randos either. OpenAI has a ton of mathematicians on staff, including Jacob Tsimerman, a fields medal winner. Scientific progress used to be people debating and correcting other people. Now it's going to be people with AI assistance debating and correcting other people with AI assistance.

          • dominotw · 3 hours ago

            > Now it's going to be people with AI assistance debating and correcting other people with AI assistance. but do these 'people' need to belong to a thriving community or not to be able to do those things?

      • verdverm · 4 hours ago

        I would say 1. This is expected if you only use a single model family like Claude, eg. we use a different model family for code review than authoring, OAI could have done this too for their math dump 2. Ai needs a good human driver beyond the trivial or mundane, they are expert enhancing machines, not expert creating machines. This is where the community comes in. Reading Tao's ChatGPT session reveals this: https://news.ycombinator.com/item?id=49010345 3. OAI is not trying to be a member of the/any community, this is not the first story to shows this, nor do I expect it to be the last. Perhaps this is them being effective altruists today? /s

    • afavour · 6 hours ago

      That’s not proof though is it? If the original LLM output is fallible surely the LLM review of that output is also very much fallible?

      • rsfern · 6 hours ago

        Proof of what? There is a sign error in one of the proofs, OpenAI acknowledged it and withdrew three papers (two relied on the result). I agree LLM review is also fallible (as is human review) but the interesting part to me is that finding this sign error before publication should have been table stakes for OpenAI, it’s their own model that found the sign error. I’m curious what was in the original prompt and what was in the prompt that led to finding the sign error, I think it matters a lot for understanding the dynamics here

      • kolinko · 5 hours ago

        Output of LLM can be infallible* even if LLMs themselves make mistakes. Ditto with humans. As much as anything can be infallible.

• caaqil · 6 hours ago

  > With automated math that community as tao pointed out is at risk. If they can be automated, they are not necessary. If they are necessary, they won't be fully automated. It's a pretty simple experiment to run, the math "community" should bear with us. Darwin would be proud.

  • doc_ick · 5 hours ago

    “If they can be automated, they are not necessary.” That’s a pretty interesting take as eventually everything could be automated.

    • spidersouris · 5 hours ago

      I guess what OP meant is automated and with 100% accuracy.

    • palmotea · 4 hours ago

      > “If they can be automated, they are not necessary.” That’s a pretty interesting take as eventually everything could be automated. It's great that we're starting to see the light at the end of the tunnel, and will some day achieve a perfect market without humans. If you think about it, all the market really needs is a people to own everything, everything else can be automated, and all those annoying human workers can be eliminated.

  • lkey · 5 hours ago

    You've abandoned every part of yourself to the siren's song of efficiency and automation, huh? To witness an arson and rejoice reveals an ugly kind of sadism.

    • falcor84 · 4 hours ago

      Will it also be "arson" if OpenAI dump on us solutions to more practical problems, for example, a blueprint for a better photolithography machine, or a nuclear power plant?

      • nancyminusone · 4 hours ago

        No that would be irony. Use up all the chip foundries and power plants to make more plans for chip foundries and power plants so we can build more chip foundries and power plants and use them to run AI to design more chip foundries and power plants.

  • kec · 5 hours ago

    This statement assumes local maxima don’t exist and greedy short term optimization always leads to long term benefit.

    • caaqil · 4 hours ago

      > assumes local maxima don’t exist and greedy short term optimization always leads to long term benefit. Even if your stated assumption was baked into the original comment, which is doubtful: the historical record shows that we will keep relearning The Bitter Lesson and each community will pretend what they do for a living is exceptional and immune because of xyz. The screams will get louder when the "greedy" and "dumb" automation comes knocking and it turns out nothing was truly immune or "nuanced ". Getting some new hobbies may be in order, it's a Brave New World.

      • kec · 3 hours ago

        "Greedy" is not a moral statement, it's a description of an optimization strategy focusing on short term improvements without a view to the long term consequences or state of the system. Bringing it back to math explicitly: you are essentially betting that the singularity is here, today, and that there are absolutely no downsides to breaking the pipeline which trains mathematicians (meaning that in 5-10 years at most there will be zero humans capable of assessing AI math output or independently advancing the state of the art).

        • caaqil · 3 hours ago

          "Who will advance [X] field or check on the work of the AI that surpassed us in 5-10 years?" is not a question that is unique to math, and the answer is pretty obvious if we get past the grieving process some are going through. The CS101 lesson in the first paragraph is appreciated, you should do it more often for us simpletons.

• thurn · 4 hours ago

  Is the risk here like "people will fund math research less because of AI"? I agree that would be bad, and we should try and stop it (along with e.g. funding for the humanities, which is in a much worse place than math!), but I'm not sure OpenAI are the right people to be mad at.

  • viccis · 4 hours ago

    Based on what I've heard from my math friends in academia, every talented undergrad who was set on going to grad school for math has switched to something like consulting internships or fintech, even if they were really passionate about math, because they don't want to spend another 7 years or so just to wind up jobless.

    • done_lurking · 4 hours ago

      This has always been the risk with majoring in math. AI is making it worse but there was never a time where studying math didn't have an extremely high opportunity cost.

      • viccis · 4 hours ago

        AI obliterating your field of study was most definitely not considered a risk with majoring in math 10 years ago.

  • svnt · 4 hours ago

    But they are the ones releasing an unverifiable (no model release) and massive and unchecked body of mathematics into the public while making exaggerated claims about its capabilities to replace human work. And they are the ones doing it in advance of a fractional sale of the company to the public.

• giancarlostoro · 4 hours ago

  It's amazing that people long long ago figured out so much math.

• nilkn · 4 hours ago

  I'm sorry but you don't actually need the mathematical community to check any of this. You just need autonomous verification, which already exists at scale and speed vastly beyond the entire human mathematical establishment. OpenAI simply rushed these out without completing that for every paper. These mistakes have nothing at all to do with the mathematical community and are 100% just the result of market pressure incentivizing speed at 1,000,000x the pace of human mathematicians. Frankly, a couple mistakes, trivially found not by humans but by humans using AI, is almost completely irrelevant. Human mathematicians have essentially nothing to contribute to this effort aside from prompting verification agents, which any of us can do if we cared to spend the time (most of us don't).

  • tylerhou · 4 hours ago

    > You just need autonomous verification, which already exists at scale and speed vastly beyond the entire human mathematical establishment. An absolutely ridiculous statement. There is a vast amount of mathematical knowledge that hasn’t even been written down, much less formalized.

    • nilkn · 4 hours ago

      There is no such thing as knowledge that has never even been written down a single time by anyone or anything. Those are simply called ideas, they are not unique to humans, and they are more likely to be wrong than right compared to anything OpenAI just produced. It would be bold and almost certainly spectacularly wrong to assume that multi-trillion-parameter AI models could not possess their own such ideas, generated during the training process as part of their internal model of the world. Beyond that, if we assume that such knowledge does exist in humans, its marginal value is clearly nearly zero now, as AI systems without access to it are vastly outperforming all human mathematicians combined by multiple orders of magnitude.

      • RunSet · 3 hours ago

        > There is no such thing as knowledge that has never even been written down a single time by anyone or anything. https://en.wikipedia.org/wiki/Tacit_knowledge#Definition

        • nilkn · 3 hours ago

          Those are simply called ideas, they are not unique to humans, and they are more likely to be wrong than right compared to anything OpenAI just produced. It would be bold and almost certainly spectacularly wrong to assume that multi-trillion-parameter AI models could not possess their own such ideas, generated during the training process as part of their internal model of the world. Beyond that, if we assume that such knowledge does exist in humans, its marginal value is clearly nearly zero now, as AI systems without access to it are vastly outperforming all human mathematicians combined by multiple orders of magnitude.

          • RunSet · 2 hours ago

            > Beyond that, if we assume that such knowledge does exist in humans, its marginal value is clearly nearly zero now "If those grapes exist they are probably sour."

            • nilkn · 26 minutes ago

              Nope. We have evidence: what human mathematicians were producing and what AI systems are now producing.

      • jansport123 · 3 hours ago

        They aren't just ideas, i read that in semiconductor manufacturing these old heads have a tremendous amount of process knowledge that's never been written down.

        • nilkn · 3 hours ago

          I'd need to see specific examples. It's not clear to me that I'd consider this knowledge about the world but rather agreed upon conventions for how humans work together.

          • tylerhou · 2 hours ago

            I do research on programming languages. There are many facts about programming languages that (1) nobody else knows and (2) I have not written down because I just haven’t had the time / opportunity. The same happens all the time in mathematics.

            • calf · 56 minutes ago

              I think his grain of truth is that human private knowledge is a) not special and AIs can/will do it too, and b) AIs will vastly outstrip human capacity for this type of knowledge.

              • tylerhou · 10 minutes ago

                lol no

      • shwaj · 3 hours ago

        This feels like a very rigid ontology. I know what type of food my dog prefers. An executive assistant has intimate knowledge of their boss’s needs. Neither of these would become knowledge if they were written down for the first time; they already are. You’re distinguishing “knowledge” from “idea” in a particular way that doesn’t correspond to common usage (see my counter examples). Without you being explicit about your definitions, I can’t tell whether what you’re saying is meaningful. It feels tautological. Given that an executive assistant has unwritten knowledge that is necessary to do their job, where your evidence that no mathematician has analogous knowledge (using the word in the common way, not whatever way you mean it)? It’s possible, but it’s not as obvious as you seem to think.

        • nilkn · 3 hours ago

          I don't know that any of this really matters. I could just as well say that you don't know anything at all about the internal experience of your dog's mind. All you can do is read behavioral patterns and attempt to make inferences. At best you have an approximation, and approximations tend to be wrong. Math is correct knowledge , not approximations of reality. Correctness is not a property of the idea or inference itself, since ideas and inferences can be wrong; it's the result of a verification process which you cannot possibly carry out in its entirety relating to the internal conscious experience of your dog. We can dig into the philosophy of these definitions, but I think the far more interesting point is that even if we grant the existence of this kind of knowledge in the minds of human mathematicians, we have passed the threshold where that "knowledge" can keep up with systems that do not have access to it. Moreover, to claim humans have a "vast amount of mathematical knowledge" that is apparently valuable and that AI systems don't have, you'd have to prove that this "knowledge" is not implied or cannot be reverse engineered from the entire corpus of mathematical writing on which AI systems are trained. You'd also have to demonstrate that this "knowledge" leads to actual results that AI systems cannot generate without it. Given the results AI systems are producing, which are far beyond human ability at this point, it is more likely that AI systems have already internalized the entirety of this so-called "tacit knowledge" and then went much further, much faster, without humans in the loop at all.

          • shwaj · 2 hours ago

            So your position boils down to “the only real knowledge is that which is provable”. Tautological. And probably not self-consistent, given Gödel.

            • nilkn · 1 hour ago

              As a response to my overall position, no, I think you threw out almost all of it. On the question of a single word out of my entire position, that is a reasonably fair statement for mathematics (which is the topic under discussion). The fact that you think it's tautological supports both that you agree with its correctness and that this is a mostly irrelevant side conversation. And whatever you think the answer should be is not somehow not subject to the constraints of logic.

      • mcswell · 28 minutes ago

        So you're saying there was no knowledge before the invention of writing?

• palmotea · 4 hours ago

  > Without a thriving mathematical community to point out these things it would have stayed broken. And that's a bad thing. If the math community didn't exist or was weak, OpenAI would still benefit from the prestige of these results they were forced to withdraw. Withdrawing these papers has harmed OpenAI's investors, and that's totally unacceptable. > With automated math that community as tao pointed out is at risk. Good to hear. The problem they represent needs to be eliminated.

• flowerlad · 4 hours ago

  This is not a problem unique to the mathematical community. What about software developer community? AI has eliminated the need for junior software engineers. Almost no one is hiring junior software engineers. But companies still need senior software engineers. Without junior engineers how will there be senior software engineers in the future? What is the solution? I don't think the solution is to say AI progress in software, mathematics etc. should be halted.

  • Centigonal · 4 hours ago

    TECH! PRIESTS! https://warhammer40k.fandom.com/wiki/Tech-Priest

  • eknkc · 4 hours ago

    AI models and tooling has advanced significantly in the last 6 months. Give it a couple of years and the companies will not need senior software engineers. They will need someone to steer the AI, maybe. Why would anyone need senior software engineers?

    • renegade-otter · 4 hours ago

      Because the models do what you ask them. "Claude, do this thing, be thorough, no mistakes" is not a good prompt.

      • CrazyStat · 3 hours ago

        Maybe for some value of "what you ask them." I have a data pipeline with 6 steps, A -> B -> C -> D -> E -> F. I asked Codex to make some specific optimizations to step B and benchmark them. It did what I asked. Then it decided to also benchmark the entire pipeline, and after noticing that step E was slow it decided to make some optimizations that I had not asked for on step E. It was at this point that I wondered why it was taking so long, saw what it was doing, and stopped it. This is GPT-6.1 Sol High.

    • mjr00 · 3 hours ago

      > Give it a couple of years and the companies will not need senior software engineers. They will need someone to steer the AI, maybe. Why would anyone need senior software engineers? Right, companies won't need software engineers. They'll just need someone who can use tools to produce source code and maintain the generated artifacts, plus make domain-specific technical decisions like "what should the system do when two users update the same record as the same time" or "how should the system behave when a message in the queue cannot be processed". We really oughta come up with a job title for these people.

      • Auracle · 3 hours ago

        What makes you think the LLM won’t just handle that on its own, or ask the person doing the “development”?

        • mjr00 · 2 hours ago

          Because there's no one correct answer and every decision involves engineering tradeoffs. Who do you think the will be choosing whether the database uses a pessimistic or optimistic concurrency strategy? The CEO?

      • threethirtytwo · 3 hours ago

        Exactly these people will be experts at typing this: “ Claude! what should the system do when two users update the same record as the same time, explain to me with full clarity” or "Claude! how should the system behave when a message in the queue cannot be processed? Give me all the possible ways ranked from best to worst, also explain to me all these concepts so I can understand as I don’t have a cs degree". If you think that there is no future where software engineers don’t matter then you are delusional. While the future is not set in stone the pace and trendline of AI point to this future as a MORE realistic future then the alternative. Your example btw is ALREADY a solved problem. AI can answer it and design around it. Agents at my company already handle our infra.

        • mjr00 · 3 hours ago

          > “ Claude! what should the system do when two users update the same record as the same time, explain to me with full clarity” This isn't even the right question to ask, I think you've basically proved my point. You are in charge of deciding what the system should do when two users update a record at the same time. It's extremely dependent on what you're trying to do. > Your example btw is ALREADY a solved problem. AI can answer it and design around it. What's the one-size-fit-all solution for concurrency management that works for every single domain and application? I'm curious. > Claude! how should the system behave when a message in the queue cannot be processed? Give me all the possible ways ranked from best to worst, also explain to me all these concepts so I can understand as I don’t have a cs degree". Who's going to make this decision? The CEO?

          • threethirtytwo · 2 hours ago

            > This isn't even the right question to ask. It’s your example. I simply took your example and asked Claude. If it’s not the right question then don’t give it out as an example. > What's the one-size-fit-all solution for concurrency management that works for every single domain and application? I'm curious. I’m curious how your brain concocted I said that. Examine the context of our conversation. What I mean there is that AI can solve those questions for every possible domain application. > Who's going to make this decision? The CEO? Armed with Claude any non technical person can make this decision.

            • mjr00 · 1 hour ago

              > If it’s not the right question then don’t give it out as an example. Knowing the right question to ask is what makes a person an engineer. > What I mean there is that AI can solve those questions for every possible domain application. Yes, if you know what to ask. You're doing an excellent job of demonstrating my point! > Armed with Claude any non technical person can make this decision. You just disproved that by asking the wrong question. Much like you, the CEO won't even know what to ask an AI.

              • threethirtytwo · 45 minutes ago

                >Knowing the right question to ask is what makes a person an engineer. Yes, but this is orthogonal to the point and that is: AI can do it too. >Yes, if you know what to ask. You're doing an excellent job of demonstrating my point! No it's your comprehension that needs work. You are missing MY point while being repeatedly getting enamored with your own point. My point is that AI KNOWS the questions. >You just disproved that by asking the wrong question. Much like you, the CEO won't even know what to ask an AI. I didn't ask a single question bro. I only regurgitated your examples. Much like AI, half your statements are based off of hallucinations.

      • eknkc · 3 hours ago

        I honestly don't think that these people will need to think in this level. `message in the queue` is an implementation detail.. I do not know or care what my if statement turned into in x86 assembly unless it becomes a performance problem and even then, I'm not profiling or debugging in machine language. Neither do most developers these days. A message in a queue becomes something akin to that in this era. I see that I got downvoted there. This is not something I advocate or look forward to but I feel this is where it is going.

        • tempfile · 3 hours ago

          It may not be necessary to think in those terms exactly, but if you are not even able to think in those terms, you probably will be no good at prompting the LLM either. It's quite plausible that someone can be a productive dev with claude if they don't know exactly how the message queue is processed. But if they don't know there is a message queue at all , that is much less likely. Likewise, you don't care exactly how an if statement gets converted into machine code, but you do know precisely what an if statement is and how it should behave, and could identify if it was buggy, and that that part of the codebase contains a bug. If you can't do that, then there is an impossible-to-estimate probability that at some point you get stuck and no progress will ever be possible. I don't see that as a winning strategy, in the long run (but it may work very well in the short term).

        • mjr00 · 3 hours ago

          > I honestly don't think that these people will need to think in this level. `message in the queue` is an implementation detail.. No it isn't lol. Have you worked on any real systems with customers? Good luck telling your boss at AWS that a poison pill message stopped the payment queue from processing so they lost $100 million in sales but hey, it's an implementation detail, no big deal.

          • eknkc · 2 hours ago

            > No it isn't lol. Have you worked on any real systems with customers? I have. Those systems already fail in spectacular ways and people tell their bosses that some worker process stopped working because its transaction IDs overflowed. I bet that sounds like `the flux capacitor stopped reticulating splines` which is already an implementation detail for the boss anyway. Nothing changes.

            • mjr00 · 2 hours ago

              Right, so who's going to ask Claude to investigate the worker processes and fix the transaction ID overflow? The CEO? Someone in marketing? The sales team? Or maaaaaybe.... an engineer ?

      • Shugyousha · 3 hours ago

        > We really oughta come up with a job title for these people. agent herder! If that comes to pass, I will have to re-evaluate my career options

    • tempfile · 3 hours ago

      What do you think "steering the AI" means? It is not like we need someone to sit at a desk and type "yes, implement it". Of course if you are a sane person you mean someone who will determine if the AI is doing "the right thing" and change its direction if it is wrong. That person is almost by definition a senior engineer.

      • eknkc · 3 hours ago

        Someone making sure that the business requirements are implemented properly and correctly. Basically a project manager. Maybe also someone testing the output and providing feedback. Not someone that says `we might have a race condition here, lets implement a distributed lock`. I think that person does not need to know about locks anymore.

        • tempfile · 2 hours ago

          The evidence from skilled vs unskilled users vibe coding is not on your side, I think.

          • eknkc · 2 hours ago

            At the moment yes, you are absolutely right. The argument was about a couple years into the future. Nobody can predict this though. I feel this will slowly steer that way. We’ll live and see.

  • jeltz · 4 hours ago

    I hope we discover a solution before innovation totally stalls, but I do not think any solution has been found yet. And I agree that halting AI is unlikely to be the solution because even if we halt the big companies China for example can still do AI. Some people hope AI will get good enough in a few years that it can innovate without human experts. Maybe? But that remains to be seen.

    • threethirtytwo · 3 hours ago

      It does remain to be seen. By the progress of AI from ChatGPT to now is horrifyingly fast. Trendlines and basic reasoning point to a most probable future where the AI is superior. We can’t just say “that remains to be seen” because the alternative is the least probable future. Anticipate the change and act prior.

  • renegade-otter · 4 hours ago

    I think this will be a self-correcting problem once the damage is done, and the only cure is more cowbell, I mean - TIME. In the age of extraction capitalism where building sustainable, profitable companies is not the goal, no one will care.

    • GuB-42 · 3 hours ago

      Hopefully, there are some sustainable, profitable companies that keep a low profile now and will take over after the "self correction" happens, and whoever invested in them will become very rich.

  • chaos_emergent · 4 hours ago

    I'm not old enough to have gone through the revolution that was programming languages that got increasingly more abstract and decoupled from the metal, but surely there's a lesson that can be learned from that era?

    • thayne · 3 hours ago

      Those abstractions are deterministic. LLMs are not.

      • convolvatron · 3 hours ago

        more importantly those abstractions were designed to try to make it easier to reason about what was going on for the author, build additional internal abstractions and to allow a reader to follow along and gain an understand of the structure. unless we believe that we can completely punt on having agency over the codebase, then llm code is only as valuable as it is readable.

        • brainwad · 3 hours ago

          The dream is that we keep agency, but also give up on reading code, by doing away with code as our level of abstraction and instead having humans edit human-readable spec files (including, say, depicting UIs directly with visual mocks). Like "no code" platforms, but for everything.

          • thayne · 2 hours ago

            If by "human readable" you mean written in natural language, then those spec files will have ambiguities and imprecisions. If you make the spec precise and unambiguous enough, then it essentially just becomes a program written in code.

          • jplusequalt · 2 hours ago

            >The dream is that we keep agency, but also give up on reading code, by doing away with code as our level of abstraction and instead having humans edit human-readable spec files (including, say, depicting UIs directly with visual mocks). Like "no code" platforms, but for everything. Human language is famously terrible at being unambiguous.

            • brainwad · 46 minutes ago

              That's ok if you and the AI are on the same page. And it is possible to write more rigorous natural language, e.g. laws are written in human languages, and yet judges and lawyers agree on how to interpret most of them and there's a defined arbitration process to resolve any new ambiguities.

      • tintor · 3 hours ago

        Human’s aren’t deterministic either.

        • recursive · 2 hours ago

          We don't keep snapshots of humans in source control.

          • latentsea · 23 minutes ago

            One time this codebase I worked on had an ASCII art image of one of the devs in it buried in a comment at the bottom of some random file. YMMV.

      • isidor3 · 2 hours ago

        Garbage collection and what the JIT decides to do often isn't.

        • jplusequalt · 2 hours ago

          I think the more salient point is that going from writing assembler to C still required you to understand a lot about your machine, algorithms, how to debug issues, and in general it demanded problem solving skills. LLMs eat away at all of these requirements.

          • isidor3 · 1 hour ago

            Perhaps, but as far as I've seen they don't make the requirement go away. They do make many types of development far more accessible, as an extension of how SQL or Excel make development far more accessible. Sure, people make messes with the tools available, and sometimes the tools can handle it and still give you something useful, many times it takes someone actually knowing what they're doing to clean it up though.

    • SoftTalker · 11 minutes ago

      Yes. As machhine power and memory increased, and as as programming abstractions advanced, software grew larger and more bloated, therefore what used to run on a single core in KB of RAM now takes multiple cores and GB of ram. The developers who knew how to write efficient low-level code found that there were no jobs for that anynmore, so today the developers who can work at that level are very few. The same will happen with AI being the new abstraction. In another decade or two, very few people will be able to write code by hand. We'll need a rack of compute in a data center and multiple KW of power to do what we used to do on a desktop PC drawing a couple of hundred Watts.

  • Insimwytim · 3 hours ago

    AI progress in software Oh, yes... the progress... You measure it by LOC, right?

  • Thanemate · 3 hours ago

    Tons of corporations rely on the fact that human interaction within software production will be as minimal as possible, and no one actually cares who trains the seniors of the future.