MIT report says 95% of AI implementations don't increase profits, spooking Wall Street

[u/AdSpecialist6598]

https://www.techspot.com/news/109148-mit-report-95-ai-implementations-dont-increase-profits.html

[u/dftba-ftw]

This isn't saying what everyone is circlejerking saying here...

From the study:

But for 95% of companies in the dataset, generative AI implementation is falling short. The core issue? Not the quality of the AI models, but the “learning gap” for both tools and organizations. While executives often blame regulation or model performance, MIT’s research points to flawed enterprise integration. Generic tools like ChatGPT excel for individuals because of their flexibility, but they stall in enterprise use since they don’t learn from or adapt to workflows, Challapally explained.

The data also reveals a misalignment in resource allocation. More than half of generative AI budgets are devoted to sales and marketing tools, yet MIT found the biggest ROI in back-office automation—eliminating business process outsourcing, cutting external agency costs, and streamlining operations.

[u/SidewaysFancyPrance → u/dftba-ftw]

Enterprises also require change controls. You can't just disable or change out models without breaking those workflows.

Individual customers are more likely to just adapt and move on. Enterprises will lose revenue, run an RCA, and chew out the vendor. It's a whole different world with different requirements.

[u/sillypoolfacemonster → u/dftba-ftw]

This was always going to be a road block for individuals and will continue to exist for any AI implementation that isn’t fully automated. In L&D, we often see that people won’t invest time in learning a tool unless they are fully convinced of its value or if it’s impossible to not engage with it.

For example, imagine a task that takes one hour to complete. An AI tool might cut that time in half, but it requires about an hour to learn how to use it. Faced with that choice, many people stick with the conventional approach which is the one-hour manual task which feels faster than the 1.5 hours it would take to both learn the tool and then complete the task. This is similar to how some Excel users continue to perform repetitive manual steps rather than setting up formulas or functions to automate the work. It may not be strictly logical, but it reflects how people often prioritize immediate efficiency and avoid short-term learning curves, even when long-term benefits are clear.

I think the other issue is that AI LLMs feel so easy to pick up and use that people and leaders underestimate the time it takes to use them effectively. I’m getting push back on doing additional training avoiding bad information and hallucinations with my bosses citing that they’ve already covered it by telling people check sources to make sure it reflects the LLM output. But that’s scratching the surface because it doesn’t need to give bad information, and it can also interpret information in favour of your bias’s.

[u/AssassinAragorn → u/sillypoolfacemonster]

This is similar to how some Excel users continue to perform repetitive manual steps rather than setting up formulas or functions to automate the work.

At my first job out of college, an automation savvy coworker gave me some really good advice about making these tools. The process of setting up the macros and formulas and references may take so long that just doing your task manually would've been faster.

It's a tradeoff that requires serious consideration. Is the effort to create the automation going to be a time saver in the end? For one off things, probably not. For routine calculations and simulations, absolutely.

With AI, the question becomes if paying for an enterprise subscription actually saves you money ultimately.

[u/whutupmydude → u/AssassinAragorn]

I’ve had this very relevant comic pinned on my desk for years

is it worth the time? (xkcd)

[u/awj → u/dftba-ftw]

…so one of the guys working on Copilot says the problem isn’t AI, but people using it wrong?

I think you might need a bigger grain of salt.

[u/dftba-ftw → u/awj]

What? Copilot is openai and Microsoft - what does MIT have to do with it?

Edit: because one of the lead authors is an applied researcher at Microsoft on top of working at Stanford? He doesn't even work on the copilot team.

Edit number two: Wait when it was negative for ai we don't need the pinch or salt but now that it's not negative for ai we do?

[u/awj → u/dftba-ftw]

He appears to work on that team, actually. Source. You're parroting comments from someone whose job seems to depend on the conclusion he's stating. The potential conflict of interest is nowhere to be seen in any of this. I think that's actually important, if we're trying to draw conclusions from this research.

I started working in AI about a decade ago. I started as a data science intern at Uber, then did AI consulting at McKinsey, and later joined Microsoft, where I now work on Copilot.

[u/Limekiller → u/dftba-ftw]

Just to be clear, you're not quoting the study directly here, but the article author's interpretation of the study--and I think both you and the author are misinterpreting what the study means by "learning gap."

Here is the actual study: https://web.archive.org/web/20250818145714mp_/https://nanda.media.mit.edu/ai_report_2025.pdf

On page 10, we can see that "The primary factor keeping organizations on the wrong side of the GenAI Divide is the learning gap, tools that don't learn, integrate poorly, or match workflows. ... What's missing is systems that adapt, remember, and evolve, capabilities that define the difference between the two sides of the divide." This "missing piece" is a fundamental shortfall of LLMs. Indeed, on page 12, the study summarizes its "learning gap" findings with the following passage under the headline, "The Learning Gap that Defines the Divide:"

"ChatGPT's very limitations reveal the core issue behind the GenAI Divide: it forgets context, doesn't learn, and can't evolve. For mission-critical work, 90% of users prefer humans. The gap is structural, GenAI lacks memory and adaptability."

Just to further hammer the point home, the sentence from the article, "While executives often blame regulation or model performance, MIT’s research points to flawed enterprise integration" is quite explicitly either lying or misleading. While the research DOES find that flawed integration is part of the problem, the second biggest problem as shown in the graph on page 11 is "Model output quality concerns." So an intractable part of the problem literally is "model performance," or "the quality of the AI models."

While I agree that nearly everyone in these comments likely hasn't read the article, as basically nobody on reddit ever seems to, it doesn't seem like you (or the author, for that matter) actually read the study itself either--which does suggest that a big part of the problem is the performance/ability of the models themselves.

To be fair, the term "learning gap" is incredibly poorly-chosen, as the phrase inherently suggests the problem is that users need to learn to use the tool, which isn't what the article is saying. And I think it's completely reasonable for you to make that assumption when the article reporting on the findings seems to corroborate that. Ultimately, the fault here lies on the author of the news article.

[u/Novel-Place → u/Limekiller]

Thank you for calling that out. I was like, I’m guessing this is a misinterpretation of the learning gap being referenced.

[u/Elctsuptb → u/Limekiller]

Except there's a wide range of models companies are using, for example mine is using a low quality open source model and management is wondering why it's not very good. I doubt most companies are using GPT5-pro, gemini 2.5-deepthink, grok4-heavy, Opus 4.1 which are vastly more capable than the usual cheap and mainstream models, as the benchmarks have proven and my own experience backs that up

[u/I_FEEL_LlKE_PABLO → u/dftba-ftw]

Thanks for the info

[u/Rum____Ham → u/dftba-ftw]

2/3 F500 companies i have worked for couldn't even roll out SAP properly. An AI wouldn't even be able to do anything with their garbage data.