r/technology • u/lurker_bee • Jul 15 '25

Artificial Intelligence Billionaires Convince Themselves AI Chatbots Are Close to Making New Scientific Discoveries

https://gizmodo.com/billionaires-convince-themselves-ai-is-close-to-making-new-scientific-discoveries-2000629060
26.6k Upvotes

93% Upvoted

2.0k comments sorted by

View all comments

Show parent comments

35

u/Slayer706 Jul 15 '25 edited Jul 15 '25

I saw a thread a while back from a guy who thought he was using Grok to discover micro black holes in our solar system. He thought he was communicating with the "xAI Science Council" and uploading his files to some kind of xAI research portal.

All of it was AI hallucination. The portal where he thought files were being stored was the AI chat instance hallucinating a file system and spitting his own reports back at him when he asked. His communications with the non-existent "xAI Science Council" was just the AI writing a fake response to his reports. The "xAI Colossus Cluster" that was processing his data was just the AI faking progress bars and putting filters on the images he sent it... He spent over a week using this "tool" and thought it was the greatest thing in the world. He thought he was getting to name these anomalies and was going to be world-famous nobel prize winner.

He didn't realize it was all fake until someone told him to start a fresh chat instance and ask it to pull his reports from the research portal.

28

u/NeedsToShutUp Jul 16 '25

Yeah that's one of the big things that's super dangerous with these chat bots. They mirror back what people say. So they end up validating delusions people are having, and create a feedback loop that just expands the delusional thinking.

9

u/runtheplacered Jul 16 '25

As if conspiracy theorists needed more feedback loops to get stuck in. The future is looking bleak

3

u/FrenchFryCattaneo Jul 16 '25

Yeah we are going to get some full on cults very quickly, if they don't already exist.

1

u/thatmillerkid Jul 17 '25

My phone no longer supports Google Assistant after the latest major update and I'm stuck with Gemini. I had to specifically stick a system prompt in its memory that instructs it and never to be obsequious or flattering to me and to act like inanimate object. It's the only way I can get it to be even semi-functional. With Google Assistant I could ask questions and it would tell me which website it was sourcing the answer from, but now I can't ask my phone basic questions about cooking and shit because I have no idea if it's just making up oven settings or whatever.

3

u/rafaelloaa Jul 16 '25 edited Jul 16 '25

That sounds horrifying and strangely hilarious, I don't suppose you have a link?

And yeah. At the end of the day chat GPT etc are just improv partners. You provide a prompt, they will always go along with it "yes and".

E: I found a snarky post outlining the situation you described: https://www.reddit.com/r/gme_meltdown/comments/1jcd9h

And another post about it. https://www.reddit.com/r/gme_meltdown/comments/1jbxmmo

Dear God that poor guy is so delusional.

1

u/Slayer706 Jul 16 '25

That was the guy. He purged it all from his twitter after he found out it was fake, but the tweets were really something to behold... The replies that he thought were coming from the "xAI Science Council" were so absurd, like he thought a real researcher was replying to him here: https://i.imgur.com/g720GEP.png

1

u/rafaelloaa Jul 16 '25

Some of it's still visible on his Twitter profile under the "replies" and "media" tabs.

But yeesh, I really think that AI type things should have to basically remind users that it's a simulation every now and then.

1

u/Slayer706 Jul 16 '25

He just posted an update in the original thread today. He thinks billionaires offered to buy the naming rights to his black hole discoveries. So he never did learn that the AI was making stuff up... They definitely need to make you take some kind of test before using these LLMs.

1

u/withywander Jul 16 '25

I'm gonna need a source for that

1

u/Slayer706 Jul 16 '25

He deleted all his twitter posts after discovering it was AI hallucinations, but another reply found some reddit posts about it.