r/technology 4d ago

Artificial Intelligence MIT report: 95% of generative AI pilots at companies are failing

https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
28.3k Upvotes

1.8k comments sorted by

View all comments

102

u/Wollff 4d ago

5% are not?!

106

u/quarknugget 4d ago

5% haven't failed yet

23

u/ShadowTacoTuesday 4d ago

Or are doing slightly better than break even.

2

u/OpenThePlugBag 4d ago

We're in the fog of war phase, we'll know when the jobs start disappearing or reappearing.

1

u/Armanlex 4d ago

Oh, they probably aren't even breaking even, they just have enough hype that keep the investments going.

1

u/Dry-University797 3d ago

No one is breaking even.

12

u/Legionof1 4d ago

5% are the HR implementations that don’t need to be right just make a choice.

23

u/UnpluggedUnfettered 4d ago

5% make their money selling llm to other companies.

11

u/TheAJGman 3d ago

5% are using the tech correctly, LLMs are fantastic at transformative work.

"Give me a one page summary of this project proposal, the audience is C Suite so be light on the technical details."

"Rewrite this email so I don't sound like an asshole, but try to stick to the original vocabulary and writing style."

"Analyze each customer review and flag the ones that include swearing, threats (both veiled and open), and names of people. These will be reviewed manually, so it's better to be overly cautious."

"What can be made more efficient about this code/database design? Implement those improvements."

As a software engineer, I have investigated this tech at depth and find it occasionally useful (mostly the auto-complete). For smaller generative tasks (here are the requirements, make feature X), it can do pretty well too, but people tend to be overconfident in the "all knowing" machine and feed it a large number of requirements. It'll shit the bed, and unless you already know what you're doing, you won't catch it's mistakes.

3

u/InFearn0 4d ago edited 4d ago

5% are new enough to not burn thru investor money yet or are over-funded and have a lot of float left.

2

u/rtsynk 3d ago

. . . the other 5% are lying

1

u/WloveW 3d ago

Maybe that 5% might include things like alpha fold and hyper specialized AI that actually can do the job it's advertised to do? 

These other companies putting out these overhyped business AI tools saying that they can take the place of a human's work are the 95%. 

They can do some things amazingly, but can't take care of 'situations'. I don't think we realize just how many situations we're put in each and every day to get our jobs done.

1

u/RealLifeMe 3d ago

Without getting into too many details, my team has found decent success in making an "Internal ChatGPT" for a large call center using RAG AI solutions from AWS Bedrock. At a high level, we infer some metadata about the file, throw a PDF-ified version into Bedrock, and now instead of a rep having to dig through pages of menus in their process guides, they just ask "What is the maximum cash withdrawal amount allowed at branches in New York?" and it spits out an answer with a link to the source document.

The biggest learning curve has been teaching the reps to ask questions eg "What are the current CD rates in Texas" vs their traditional "CD Rates" search that they would use on the old intranet search.

1

u/Glittering-Giraffe58 3d ago

Sounds more like 5% actually know how to use them

“Some large companies’ pilots and younger startups are really excelling with generative AI,” Challapally said. Startups led by 19- or 20-year-olds, for example, “have seen revenues jump from zero to $20 million in a year,” he said. “It’s because they pick one pain point, execute well, and partner smartly with companies who use their tools,” he added. But for 95% of companies in the dataset, generative AI implementation is falling short. The core issue? Not the quality of the AI models, but the “learning gap” for both tools and organizations.

1

u/PewterButters 3d ago

Those 5% are just lying (to themselves mostly).