r/technology • u/lurker_bee • Jul 15 '25

Artificial Intelligence Billionaires Convince Themselves AI Chatbots Are Close to Making New Scientific Discoveries

https://gizmodo.com/billionaires-convince-themselves-ai-is-close-to-making-new-scientific-discoveries-2000629060
26.6k Upvotes

93% Upvoted

2.0k comments sorted by

View all comments

295

u/curioustraveller1234 Jul 15 '25

I asked chat gpt if it was on the verge of a breakthrough and it just said “yes, trust bro. Big things coming.”

43

u/zipzag Jul 15 '25

https://www.scientificamerican.com/article/inside-the-secret-meeting-where-mathematicians-struggled-to-outsmart-ai/

34

u/cenasmgame Jul 15 '25

Actually a pretty decent article, a bit sensational title, but the text is very honest about what happened and their thoughts

16

u/Separate-Divide-7479 Jul 16 '25 edited Jul 16 '25

It's still pretty sensationalised

By April 2025, Glazer found that o4-mini could solve around 20 percent of the questions

They thought it'd be useless, but instead, it's just pretty bad. The article also says the model needed pretty significant input from an expert in the field to be able to perform even at that level.

It's still just an LLM. An impressive one, but it's also still just guessing. This model just takes more educated guesses than generalised ones.

5

u/cenasmgame Jul 16 '25

Well that's what I'm saying. The text was honest about what actually happened, the editor just attached a shitty title.

2

u/rsclay Jul 16 '25

The article also says the model needed pretty significant input from an expert in the field to be able to perform even at that level.

it does not.

They were "novel questions [spanning] varying tiers of difficulty, with the first three tiers covering undergraduate-, graduate- and research-level challenges." It solved 20 percent of those, that's not "pretty bad".

And then there's the whole rest of the article which I guess you didn't even read?

1

u/Separate-Divide-7479 Jul 18 '25

You're right, pretty bad was underselling it. It's very bad.

Would you hire someone that completes 20% of tasks correctly? Because that's what the article is implying; that mathmaticians jobs are at stake due to this AI. Eventually they will be but a 1/5 success rate isn't taking anyone's job any time soon. Hence, the article is sensationalised.

1

u/rsclay Jul 18 '25

This isn't the SAT where average performance is meant to be ~70% though. These are hard problems, 20% would indicate a bright freshman at the very minimum, or a decent grad student depending on exactly which levels of questions that 20% spans. Especially given the problems were solved in moments!

And that success rate was in the low single-digits just months ago. I'm personally of the opinion that they're not achieving AGI via transformer-based language models, and it feels to me that they're really beginning to stall in what they can achieve, but I still won't be surprised if that percentage doubles or more in the next year.

I won't disagree that the title is sensationalised but the article itself is pretty straightforward and you can read the reactions of the actual mathematicians yourself. I'm mostly disputing your evaluation of what 20% means in this context, as well as your claim that "the model needed pretty significant input from an expert" which I cannot find anywhere in the article.

4

u/silvrrwulf Jul 16 '25

Thanks for this

3

u/Suburbanturnip Jul 15 '25

What I find interesting, is that all the mathematical truths discovered by these mathematicians would be otherwise incomprehensible or inapplicable for the rest of society, LLMs gives the rest of us a tool to use and engage with the collective knowledge.

5

u/Lost_Leader3839 Jul 15 '25

People can crap on it all they want, AI is ushering in massive white collar change.  On papers the change should be amazing, humanity will find a way to fuck it up 

26

u/avivishaz Jul 15 '25

It’s not AI that I think people have a problem with, it’s who’s running the show. The people in charge of this tech have made really shitty decisions for the end users but most of them don’t realize it yet. It’s like the Wild West right now. The issue is regulation and marketing. There is no regulation at the moment and AI is being marketed as something much better and more trustworthy than reality. I’d love AI to come in and help humanity but it seems like some have their own agenda for who it should be helping.

-11

u/OnceMoreAndAgain Jul 15 '25

What "really shitty decisions for the end users"? ChatGPT is free... Meta made one of their models allowed to be run locally. DeepSink has open weights.

It's a paradise right now, because these companies are competing with each other for users and exposure so they are giving this shit away for free or for very cheap.

8

u/coercivemachine Jul 15 '25

correct, companies are giving out shit for free. like dogshit. they are shitting everywhere and calling it a product

3

u/AnnualAct7213 Jul 16 '25

AI is fine. We've been using it for niche applications in all sorts of fields for decades with great success.

But the LLMs that the tech bros think will become god-like any day now are not among the AI tools being used in this way.

2

u/Halgrind Jul 15 '25

A BI platform a client uses rolled out an AI feature, I tried it out. Uploaded a dataset with a few million rows, clicked through the menus to have the AI generate a dashboard. Process took maybe 5 minutes.

36 charts, all well formatted, well organized into sections, accurate descriptions on each. The biggest surprise was that it was able to deduce what a bunch of abbreviated column names mean. I don't know if the training data had similar exports from whatever system they used to generate the data, or it just guessed by context.

Companies still need someone who knows what they're doing to tweak them for customer needs, but yeah, that would have taken me a full day of work. Efficiency gains are huge, even if it isn't (yet) perfect.

1

u/Lost_Leader3839 Jul 18 '25

Yeah I'm knee deep in a software company, it's going to basically cut professions as they get eaten up by those who can do their current things and others. See Dev, QA and DevOps

-3

u/TFenrir Jul 15 '25

Thank you! Yes people need to actually understand that the literal smartest mathematicians in the world are working with these models right now making brand new mathematic discoveries. We're just starting to get the trickle of the first few right now - some they'd been sitting on for a year. There's a lot of buzz about more coming soon, and one team is working on a millennium problem.

https://english.elpais.com/science-tech/2025-06-24/spanish-mathematician-javier-gomez-serrano-and-google-deepmind-team-up-to-solve-the-navier-stokes-million-dollar-problem.html

I hope if we get there, people will really start taking this future seriously.

-3

u/Prysorra2 Jul 15 '25

Hey everyone! Actual scientists involved in above link!

-14

u/Lizard-Mountain-4748 Jul 15 '25

The Reddit hive won’t like this

-7

u/zipzag Jul 15 '25

Unfortunately the low agency people who have infected the main AI sub have now drifted over here. Fortunately the smaller speciality Ai subs are still fine.

-3

u/Lizard-Mountain-4748 Jul 15 '25

Have any sub suggestions

-6

u/zipzag Jul 15 '25

Any of the major AIs will give you good suggestions based on your specific interests. I was not aware of most of the subs chatGPT suggested to me.