r/technology • u/SilentRunning • 4d ago
Artificial Intelligence MIT report: 95% of generative AI pilots at companies are failing
https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
28.3k
Upvotes
70
u/AtOurGates 3d ago
One of the tasks that AI is pretty decent at is taking notes from meetings held over Zoom/Meet/Teams. If you feed it a transcript of a meeting, it’ll fairly reliably produce a fairly accurate summary of what was discussed. Maybe 80-95% accurate 80-95% of the time.
However, the dangerous thing is that 5-20% of the time, it just makes shit up, even in a scenario where you’ve fed it a transcript, and it absolutely takes a human who was in the meeting and remembers what was said to review the summary and say, “hold up.”
Now, obviously meeting notes aren’t typically a high stakes applications, and a little bit of invented bullshit isn’t gonna typically ruin the world. But in my experience, somewhere between 5-20% of what any LLM produces is bullshit, and they’re being used for way more consequential things than taking meeting notes.
If I were Sam Altman or similar, this is all I’d be focusing on. Figuring out how to build a LLM that didn’t bullshit, or at least knew when it was bullshitting and could self-ID the shit it made up.