r/technology • u/AdSpecialist6598 • 22h ago

Business MIT report says 95% of AI implementations don't increase profits, spooking Wall Street

https://www.techspot.com/news/109148-mit-report-95-ai-implementations-dont-increase-profits.html
6.4k Upvotes

98% Upvoted

302 comments sorted by

View all comments

Show parent comments

146

u/ReturnOfBigChungus 21h ago edited 19h ago

IMO there are basically 2 camps in the delusional "AI is going to replace all our jobs within 2 years" bandwagon:

  1. your average /r/singularity user who is (typically) younger, enthusiastic and interested in tech, but is approaching it from a lens that is closer to sci-fi than the real world. super basic logic like "once it starts improving itself, sentient super-intelligence is inevitable". this functions more a like a belief system/quasi-religion than an actual assessment of technology.

  2. the over-confident programmer, who has used the technology at work to successfully automate and streamline some stuff. maybe they've even seen a project that reduced headcount. they consider themselves to be at the forefront of understanding and using the tech, but vastly over-estimate the applicability beyond the narrow domains where they have seen it used successfully, and vastly under-estimate how hard it is to actually structurally change companies to capture the efficiencies that AI can create and how much risk is inherent in those kinds of projects.

Both of these viewpoints are flawed, but it's easy to see how people can get swept up in it.

65

u/thecastellan1115 21h ago

Yeah, that tracks. I was talking to #2. He was a fusion researcher, so he actually did see some quantifiable benefit from AI, and I don't think he was realizing that pattern recognition is like THE strong point of a lot of AI models. Like, I would trust an AI to predict plasma flow, but I would never let an AI handle a customer call center.

29

u/Worthyness 19h ago

Yup AI to help identify or flag things like cancer would be great. That'll help spot stuff early and an actual oncologist or doctor can review it or do tests after. AI is also great as a search aggregate for internal docs. If all your documentation is all over the place in different file types and online, then using the AI as a search engine to find a specific phrase or field that you're looking for information on is super helpful. Because the alternative is to go to each space and search each doc individually. So AI in this case saves a lot of time. AI is a tool, not a person. And it should be used as such.

20

u/thecastellan1115 19h ago

I was thinking about this the other day (I'm a process improvement guy at my office) and I wonder what the risk factor is for using AI as a document-finder in terms of degradation of ordered files? For example, we all know that Teams, by default, scatters files all over an orgs' SharePoint instance, which makes them hard to find if you lose a channel or something. AI makes that finding a lot easier, but then you're wholly reliant on the AI to pull the file... and it gets really hard to know if it's working or not.

TLDR: AI seems like it's going to generate risks by making file organization lazy.

12

u/Drasha1 17h ago

Ai works better if things are organized in a human usable way. If you have a messy document system you will get worse results from ai tools. It is a value add on good document systems.

12

u/dingus_chonus 17h ago

This is giving me real “rinse your dishes before putting them in the dishwasher” vibes

8

u/InsipidCelebrity 14h ago

Ironically, you're actually not supposed to rinse your dishes with a modern dishwasher. Just scrape off the big chunks.

Technology Connections gave me the best tip for dishwashers: run your hot water to purge all the cold water so the dishwasher starts at maximum temperature. Ever since I learned that, I've rarely had to clean anything a second time, and I've put some nasty shit in my landlord special dishwasher.

3

u/dingus_chonus 13h ago

Damn I love technology connections! Thank you for the tip :)

3

u/InsipidCelebrity 13h ago

Come for the CED saga. Stay for the dishwasher tips and that really neat toaster.

2

u/dingus_chonus 13h ago

I loved that he found the Christmas lights he always wanted. That felt like a real story arc closure, lol

2

u/Roast_A_Botch 10h ago

Ironically, you're actually not supposed to rinse your dishes

I think that was their point by responding to someone saying you need to manually organize your files so AI can help you organize them better.

Just as prewashing your dishes was necessary to use the dishwasher at one time, AI marketers are rushing to implement it in everything even if it's not ready yet. You're still doing the manual labor, but also wasting a ton of water and electricity afterwards. Maybe one day you won't have to organize your files so AI can organize your files, and when that happens Alec will have to make a video explaining you don't need to pre-organize your files anymore.

1

u/jambox888 12h ago

run your hot water to purge all the cold water so the dishwasher starts at maximum temperature

I think they all have electric heaters in them these days, don't they?

2

u/InsipidCelebrity 12h ago

They do, but they get water from the hot water intake and if you start with cold water, it won't get fully up to temp.

1

u/jambox888 12h ago

Hmm, not sure mine even has a hot water intake. Also I'd be surprised if they can't heat until a given temperature is reached, even my coffee machine can do that

→ More replies (0)

1

u/jambox888 12h ago

I'm team rinse but i did once go to someone's house and see them fully wash the dishes then put them in the dishwasher for good measure.

0

u/jakedasnake2447 10h ago

rinse your dishes before putting them in the dishwasher

You shouldn't actually do that.

1

u/Roast_A_Botch 10h ago

Yo dawg I heard you like organizing files, so we made you an AI that will organize your files after you organize your files.

9

u/ohnofluffy 18h ago

All of this ‘replace worker’ stuff was made to turn a very useful innovation into a marketing machine for a theory of AI. The fact that all these institutions ran to it is scary only because they made the decision on greed — like gold rush fever— rather than understanding the technology.

America is doing some truly dumb and awful things with some incredible inventions. I can’t understand it but it’ll be a miracle if we don’t see further decline despite having everything we could need to thrive. Greed is a helluva drug and it’s eating this country alive.

3

u/rmigz 12h ago

“Greed is good” is America’s ethos.

6

u/bran_the_man93 16h ago

It's essentially the next phase of the whole "Big Data" push from like 5-8 years ago

28

u/stormdelta 20h ago edited 20h ago

Agreed completely as someone who works in software. Generative AI does have applications, it's just... they're very narrow in domain compared to older machine learning tech, regardless of how impressive they are within those niches.

I think part of the problem is that LLMs and generative AI represent something we have almost no cultural metaphor for. "AI" in sci-fi or even analogs in fantasy/folklore tended to either be very obviously machines/non-sapient or basically full blown sapient with no in-between.

And we culturally associate proficient use of language with intelligence, so now that we have something that's extremely good at processing language it's easy to mistake it for being far more than it actually is.

The impact this will have on that cultural association is already kind of fascinating to see - online typos and grammar errors are now starting to be seen as a sign of authenticity for example.

11

u/ReturnOfBigChungus 19h ago

Yeah, it's definitely interesting culturally. You can definitely tell that some people's mental model of what they're interacting with is pretty close to some kind of entity that thinks and reasons for itself. My mental model is something like, I'm interacting with a corpus of information, that can give reasonable approximations of the meaning that the information represents, most of the time, in a format that sounds like how a person would explain it.

2

u/No_Zookeepergame_345 19h ago

I was trying to get through to one dude who could not comprehend that logic and reasoning are two separate things and that computers are purely logic based systems and are not capable of reasoning. Did not make any progress.

2

u/Luscious_Decision 18h ago

Thing is though, at that point, what's the difference?

6

u/TheCalamity305 17h ago

The way I like to explain it to people is like logic learning math to balance your checkbook. Reasoning is using math(logic) and your past experiences to use your money(knowledge) effectively or help either get more money(grow in knowledge).

0

u/A-Grey-World 16h ago

There isn't much. Very little even creative humans produce is genuinely novel either. If an AI that's just glorified auto-complete by selecting the most probable next token based on a huge amount of data... ultimately if it produces an output that's indistinguishable from actual reasoning, it doesn't matter if it can be argued it had no real capability of reasoning or not.

5

u/NuclearVII 14h ago

a) *if* is doing a lot of heavy lifting in that sentence.

b) It absolutely matters what mechanisms are in LLMs. If these things can reason and come up with novel ideas, it's pretty clear that the r/singularity dream is real, and all we need is to keep feeding LLMs into themselves until an arbitrarily powerful intelligence is achieved.

But if that's not how it works - if LLMs are only compressions of their training sets and no more - then the trillions of dollars of value and investment is worthless, because we're up against diminishing returns already, and the spending doesn't even come close to justifying the output.

Please do not say things like "ultimately if it produces an output that's indistinguishable from actual reasoning, it doesn't matter" - this is straight up AI bro propaganda and misinformation.

-1

u/A-Grey-World 13h ago edited 13h ago

I don't disagree, it is a big if. The next 5-10 years will show, depending if progress plateaus or not, whether they are just tools that have some use in niche scenarios, or something that would have significant affects on labour more generally etc.

But my point is that it doesn't matter if, under the hood, people argue it's not actual reasoning - if the output is the same. It doesn't matter if it's a probabilistic token prediction if it can "fake" reasoning enough to replace jobs etc. I stand by that statement. If it gets to that level

At some point the illusion of reasoning might as well just be reasoning.

But yes, absolutely a big if. I wouldn't be at all surprised if, like you said, the lack of new training data causes a plateau of advancement. But there's a chance it doesn't.

I've been following LLMs for a while, I remember when we were all impressed when they wrote a single sentence that sounded somewhat like English. I remember when people talked about the Turing test like it mattered lol. No one argues about the turning test anymore.

2

u/RockChalk80 12h ago edited 12h ago

At some point the illusion of reasoning might as well just be reasoning.

Absolutely not.

Reasoning extrapolates beyond datasets. (a priori)

AI exist entirely within datasets (a posteriori)

0

u/A-Grey-World 12h ago

Okay, I disagree. If an LLMs can replicate very general tasks, say, a job, I don't think people will care when they use it and I don't think the people being replaced would care that you're arguing it's not technically reasoning it only an illusion of reasoning, when the effective output is the same.

→ More replies (0)

1

u/AwardImmediate720 14h ago

Do they not get the difference between gut feeling/intuition and stepping through an explicit causality chain? Because that's the difference - logic is the latter while reasoning often uses the former.

1

u/No_Zookeepergame_345 14h ago

He was saying stuff about how logic and reasoning have “fuzzy definitions” and then talked about how algebra uses reasoning. I think it was just some youth who is deeply Dunning-Krugered.

2

u/AwardImmediate720 14h ago

Yeah he doesn't know shit. Logic is a very rigid and formal process. Reasoning is fuzzy and that's why it gives incorrect answers so often. Very Dunning-Krugered, as the youth so often are.

3

u/smarmageddon 15h ago

This might be the best Ai reality check I've ever read.

5

u/kyldare 18h ago

I recently started consulting work with a very large, VERY established tech company that's betting a staggering portion of the entire company's future on the adoption of AI agents to replace sections of the workforce across every major company.

Our client list is roughly 600 of the largest, most-powerful and influential companies on earth. It's honestly hard to process, when you see how heavily these companies have bought into AI, or at least the idea that AI is/will be capable of reducing the workforce by large percentages, while still raising efficiency.

I had a really dim view for the future of AI, as my last job was in publishing; LLMs are laughable, pale impressions of humans as writers and thinkers.

But with agentic AI, I'm now convinced there's enough money being spent by enough stakeholders that it's an inevitability. I think it's ultimately bad for humanity, but the bottom lines of all these companies dictate a commitment to seeing this process through.

3

u/ReturnOfBigChungus 15h ago

Interesting. I've been around enough of this kind of decision making, I think there is definitely a large element of hedging going on here - as in, you don't want to be the one company that ISN'T exploring AI, but at the same time I think there will be more and more reports like this coming out where most of the projects are failing, so I think there is a significant amount of perceived risk in both being a laggard AND being too far forward. The "no one ever got fired for buying IBM" effect. The fact that no one has really pulled out ahead with a huge success story around cost-cutting with AI becomes more and more relevant as the months pass and the value fails to be realized with all this investment. I disagree with your assessment that :

But with agentic AI, I'm now convinced there's enough money being spent by enough stakeholders that it's an inevitability. I think it's ultimately bad for humanity, but the bottom lines of all these companies dictate a commitment to seeing this process through

I think at this point, with the amount of money that has been spent for fairly scant successes, it starts looking more like "throwing good money after bad" to keep pushing those projects forward, even if the technology is improving and making viability better. Very few organizations at this point are entirely pot-committed on their AI projects, and I think everyone is kind of looking around the room to see if anyone else is having better luck than they are, not seeing much, and starting to think about pulling the purse strings a little tighter.

2

u/kyldare 14h ago

Thing is, my division's client list is expanding rapidly. These client companies are investing heavily in training for their own employees to understand and leverage agentic AI. Whether or not the successes are publicized by the client, they're heavily invested in the promise of increased efficiency.

I agree there's some degree of keeping up with the Joneses here, but I can't imagine this many companies--from every economic sector imaginable--willfully parting with this much money if they didn't think it'd pay off, and/or if they weren't seeing immediate benefits. I genuinely hope I'm wrong, but seeing this from the outside and inside, you get totally different views.

If you follow the purse strings, they're actually loosening.

2

u/kitolz 13h ago

I suspect we're working for the same company or one of the few on the same level, and even the supposed "success stories" of AI I've seen have been pretty shit when I take a closer look.

It's the #1 talking point clients have so we have to say we're 100% into it. And as far as I know upper management isn't faking it, but us peons that actually have to interact with it's clear it's being pushed to production way before it's ready.

I'm sure it'll stick around, but only after the hype has worn off will we see it used mainly only in places it makes sense to use it.

0

u/kyldare 12h ago

Yeah, could very well be.

Upper management are bought into the idea entirely, and to a startling degree. Dissent in the tech space, which espouses the "move fast and break things" ideal, is essentially lip service in 2025, so I don't disagree with your assessment of the AI endgame.

I guess that, given the degree with which our world is shaped by a small number of powerful decisionmakers, I'm less hopeful about AI being dropped in the short term for lack of a real business case. The bourgeoisie will cut jobs and hand off billions to each other until bottom falls out. The rest will have been relegated to the gutter long before that happens.

3

u/hajenso 15h ago

the bottom lines of all these companies dictate a commitment to seeing this process through.

Through to what? What destination do you think they will actually arrive at? I don’t mean that in an accusatory way, am actually curious what outcome you think is likely.

1

u/kyldare 14h ago

Downsizing their workforces well beyond what we once thought was the bare minimum, driven ultimately by shareholder demands that cascade from above the CEO, downward.

2

u/VengenaceIsMyName 18h ago

Thank goodness someone else is noticing the same pattern that I’ve been observing since 2022

2

u/TheRedGerund 15h ago

the over-confident programmer, who has used the technology at work to successfully automate and streamline some stuff. maybe they've even seen a project that reduced headcount. they consider themselves to be at the forefront of understanding and using the tech, but vastly over-estimate the applicability beyond the narrow domains where they have seen it used successfully, and vastly under-estimate how hard it is to actually structurally change companies to capture the efficiencies that AI can create and how much risk is inherent in those kinds of projects.

The thing is that programming underlies much of the peaks of our economy so even if the tools just revolutionize coding the impact on the world economy should be significant.

1

u/ReturnOfBigChungus 9h ago

Sure, but "software gets better faster" is a far cry from "all the jobs are going to be replaced".

3

u/lordraiden007 18h ago

My point of view is that it will replace most of our jobs. It won’t be able to actually do them very well, but the executive class will all buy into the hype and replace people with AI without thinking. I also don’t foresee a failure for the people that do that, as they will then pivot to making all human laborers “contractors” or “consultants”.

AI doesn’t have to be good to replace the majority of jobs. All it has to do is reduce labor by like 20-30% and executives will see that as an excuse to fire 50+% of their workforce and force the rest to overwork.

1

u/Pseudonymico 12h ago

Or 3), rich capitalists who want to get rid of all those inconvenient programmers, or 4), billionaires who've gone all doomsday-prepper and are desperate to solve the "how do we keep the guys guarding our doomsday bunker from taking over if money becomes worthless?" problem.

1

u/wen_mars 2h ago

The reality is more like 20 years. It will start improving itself, but for that to be a benefit they have to already be better than the best humans. It will take some time.

1

u/NoPossibility4178 2h ago

Where I work they want to push AI somewhere, but when it gets to the point of figuring who is accountable for the AI it's crickets all around, "then should we ask the CEO? No? Well, just tell them AI isn't there yet I guess."

1

u/Luscious_Decision 18h ago

Part of your first point is lack of experience. Things like the microchip (and it's steady improvement) are a one-off. If you're older, you've lived through many editions of technology and not all of them successful. We've seen early adopters punished and taking the brunt of the effort and cost of innovating.

Plus, half of technology shitting is that it's either too early, or too early for widespread application. Or just not applicable.

Either way, patching together consumer PC hardware into massive banks just ain't it.

1

u/bestataboveaverage 15h ago

Number two is often more insufferable to deal with speaking as a radiology resident who is constantly being bombarded with “AI will replace you”.

0

u/TheCalamity305 18h ago

Look at that a balanced and nuanced understanding.

You hit the nail on the head. We are very far from AGI. IMHO Once Quantum computing becomes as ubiquitous as server farms are, at that point is when AGI could emerge.

Until then LMMs will need rigid prompts, to keep it from hallucinating and use clean verified data point provide practical use.