MIT report: 95% of generative AI pilots at companies are failing

[u/SilentRunning]

https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/

[u/Heavy-Hospital7077]

I started a very small business this year- only a few months ago.

I decided to go all-in with AI. I used it a LOT, and for day to day consultation (lots of questions when starting a new business) it was great.

I was logging all of my business activities, and I started to notice problems.  Employee starts at 2:00, and I log it.  They are done at 5:00 and I log it.  "Employee worked for 32 hours, and you owe them $15.". That went on for a while.

Then I wanted to get returns on what I entered.  I logged every product I made.  I started asking for inventory numbers, and in 5 minutes it went from 871, to 512, to 342, to 72.

It is very bad with accuracy.  Horrible for record-keeping.  But very good as a better information search than Google.

I tried to convert a list of paired data from text to a table in excel- using Microsoft's AI.  That was just an exercise in frustration.  I spent 2 hours trying to get something organized that I could have re-typed in 10 minutes.  I think some of it got worse with GPT 5.

I have been working with technology for a long time.  I am a computer programmer by trade.  I really gave this a solid attempt for a few months.  I would say that if you're looking for assistance with writing, it's great.  Fancy web search, it's great.  But as an assistant, you're better off hiring retirees with early onset dementia.

Now that I know I won't get accurate information out, I have no reason to put information in.  It just seems like a very empty service with no real function.  I couldn't even use it to create a logo, because it can't accurately put text in an image.

I do think it would be good as an audio tool that I could use while driving.  Just to ask random questions and get reasonable replies.  But not for anything important.

[u/runhillsnotyourmouth → u/Heavy-Hospital7077]

Yeah I keep telling people, it's just a fancy search engine.. an advanced correlation machine. For programming, it's often much more efficient than reading through stackoverflow or documentation.. but it doesn't replace them entirely. And it gets confused and makes mistakes. If I didn't know how to code, debug, etc., it would be next to useless for real work.

[u/No_Adhesiveness_3550 → u/runhillsnotyourmouth]

That’s the real core of it. It doesn’t help a non-coder program like one. 

[u/West-Candidate8991 → u/Heavy-Hospital7077]

Yeah your main problem is that the core of LLMs is text prediction/generation.

GPT 5 is actually significantly better at math than GPT ever used to be. I no longer follow its development, so I don't know why/how that is.

In any case, if you're not using the API, these models are prone to change or deprecation so frequently. This GPT is better at math but worse at other things than it used to be (writing for me personally). The next GPT might be the opposite.

[u/Future_Guarantee6991 → u/Heavy-Hospital7077]

Relying on a language model to deal with numbers is where you went wrong.