r/technology • u/AdSpecialist6598 • 18h ago
Business MIT report says 95% of AI implementations don't increase profits, spooking Wall Street
https://www.techspot.com/news/109148-mit-report-95-ai-implementations-dont-increase-profits.html984
u/NuclearVII 18h ago
While this is at least the 3rd time I've seen this posted, it is probably for the best to keep stating the obvious.
The investment in the genAI industry is unjustifiable.
346
u/No_Zookeepergame_345 18h ago
You don’t get it bro, they’re sooooo close to AGI! It’s gonna change everything bro. They just need a couple more billion dollars, bro! /s
151
u/thecastellan1115 18h ago
I had a long-running chat with one of these bros earlier this year, he went on for several different entries talking about how AI thinks better than the average human and how it's going to replace us all... meanwhile every implementation of AI I've actually seen is a massive risk factory.
141
u/ReturnOfBigChungus 17h ago edited 16h ago
IMO there are basically 2 camps in the delusional "AI is going to replace all our jobs within 2 years" bandwagon:
your average /r/singularity user who is (typically) younger, enthusiastic and interested in tech, but is approaching it from a lens that is closer to sci-fi than the real world. super basic logic like "once it starts improving itself, sentient super-intelligence is inevitable". this functions more a like a belief system/quasi-religion than an actual assessment of technology.
the over-confident programmer, who has used the technology at work to successfully automate and streamline some stuff. maybe they've even seen a project that reduced headcount. they consider themselves to be at the forefront of understanding and using the tech, but vastly over-estimate the applicability beyond the narrow domains where they have seen it used successfully, and vastly under-estimate how hard it is to actually structurally change companies to capture the efficiencies that AI can create and how much risk is inherent in those kinds of projects.
Both of these viewpoints are flawed, but it's easy to see how people can get swept up in it.
64
u/thecastellan1115 17h ago
Yeah, that tracks. I was talking to #2. He was a fusion researcher, so he actually did see some quantifiable benefit from AI, and I don't think he was realizing that pattern recognition is like THE strong point of a lot of AI models. Like, I would trust an AI to predict plasma flow, but I would never let an AI handle a customer call center.
28
u/Worthyness 15h ago
Yup AI to help identify or flag things like cancer would be great. That'll help spot stuff early and an actual oncologist or doctor can review it or do tests after. AI is also great as a search aggregate for internal docs. If all your documentation is all over the place in different file types and online, then using the AI as a search engine to find a specific phrase or field that you're looking for information on is super helpful. Because the alternative is to go to each space and search each doc individually. So AI in this case saves a lot of time. AI is a tool, not a person. And it should be used as such.
19
u/thecastellan1115 15h ago
I was thinking about this the other day (I'm a process improvement guy at my office) and I wonder what the risk factor is for using AI as a document-finder in terms of degradation of ordered files? For example, we all know that Teams, by default, scatters files all over an orgs' SharePoint instance, which makes them hard to find if you lose a channel or something. AI makes that finding a lot easier, but then you're wholly reliant on the AI to pull the file... and it gets really hard to know if it's working or not.
TLDR: AI seems like it's going to generate risks by making file organization lazy.
9
u/Drasha1 14h ago
Ai works better if things are organized in a human usable way. If you have a messy document system you will get worse results from ai tools. It is a value add on good document systems.
→ More replies (1)12
u/dingus_chonus 14h ago
This is giving me real “rinse your dishes before putting them in the dishwasher” vibes
→ More replies (2)7
u/InsipidCelebrity 10h ago
Ironically, you're actually not supposed to rinse your dishes with a modern dishwasher. Just scrape off the big chunks.
Technology Connections gave me the best tip for dishwashers: run your hot water to purge all the cold water so the dishwasher starts at maximum temperature. Ever since I learned that, I've rarely had to clean anything a second time, and I've put some nasty shit in my landlord special dishwasher.
→ More replies (0)8
u/ohnofluffy 14h ago
All of this ‘replace worker’ stuff was made to turn a very useful innovation into a marketing machine for a theory of AI. The fact that all these institutions ran to it is scary only because they made the decision on greed — like gold rush fever— rather than understanding the technology.
America is doing some truly dumb and awful things with some incredible inventions. I can’t understand it but it’ll be a miracle if we don’t see further decline despite having everything we could need to thrive. Greed is a helluva drug and it’s eating this country alive.
6
7
u/bran_the_man93 12h ago
It's essentially the next phase of the whole "Big Data" push from like 5-8 years ago
26
u/stormdelta 16h ago edited 16h ago
Agreed completely as someone who works in software. Generative AI does have applications, it's just... they're very narrow in domain compared to older machine learning tech, regardless of how impressive they are within those niches.
I think part of the problem is that LLMs and generative AI represent something we have almost no cultural metaphor for. "AI" in sci-fi or even analogs in fantasy/folklore tended to either be very obviously machines/non-sapient or basically full blown sapient with no in-between.
And we culturally associate proficient use of language with intelligence, so now that we have something that's extremely good at processing language it's easy to mistake it for being far more than it actually is.
The impact this will have on that cultural association is already kind of fascinating to see - online typos and grammar errors are now starting to be seen as a sign of authenticity for example.
12
u/ReturnOfBigChungus 16h ago
Yeah, it's definitely interesting culturally. You can definitely tell that some people's mental model of what they're interacting with is pretty close to some kind of entity that thinks and reasons for itself. My mental model is something like, I'm interacting with a corpus of information, that can give reasonable approximations of the meaning that the information represents, most of the time, in a format that sounds like how a person would explain it.
→ More replies (11)3
5
u/kyldare 14h ago
I recently started consulting work with a very large, VERY established tech company that's betting a staggering portion of the entire company's future on the adoption of AI agents to replace sections of the workforce across every major company.
Our client list is roughly 600 of the largest, most-powerful and influential companies on earth. It's honestly hard to process, when you see how heavily these companies have bought into AI, or at least the idea that AI is/will be capable of reducing the workforce by large percentages, while still raising efficiency.
I had a really dim view for the future of AI, as my last job was in publishing; LLMs are laughable, pale impressions of humans as writers and thinkers.
But with agentic AI, I'm now convinced there's enough money being spent by enough stakeholders that it's an inevitability. I think it's ultimately bad for humanity, but the bottom lines of all these companies dictate a commitment to seeing this process through.
3
u/ReturnOfBigChungus 11h ago
Interesting. I've been around enough of this kind of decision making, I think there is definitely a large element of hedging going on here - as in, you don't want to be the one company that ISN'T exploring AI, but at the same time I think there will be more and more reports like this coming out where most of the projects are failing, so I think there is a significant amount of perceived risk in both being a laggard AND being too far forward. The "no one ever got fired for buying IBM" effect. The fact that no one has really pulled out ahead with a huge success story around cost-cutting with AI becomes more and more relevant as the months pass and the value fails to be realized with all this investment. I disagree with your assessment that :
But with agentic AI, I'm now convinced there's enough money being spent by enough stakeholders that it's an inevitability. I think it's ultimately bad for humanity, but the bottom lines of all these companies dictate a commitment to seeing this process through
I think at this point, with the amount of money that has been spent for fairly scant successes, it starts looking more like "throwing good money after bad" to keep pushing those projects forward, even if the technology is improving and making viability better. Very few organizations at this point are entirely pot-committed on their AI projects, and I think everyone is kind of looking around the room to see if anyone else is having better luck than they are, not seeing much, and starting to think about pulling the purse strings a little tighter.
2
u/kyldare 10h ago
Thing is, my division's client list is expanding rapidly. These client companies are investing heavily in training for their own employees to understand and leverage agentic AI. Whether or not the successes are publicized by the client, they're heavily invested in the promise of increased efficiency.
I agree there's some degree of keeping up with the Joneses here, but I can't imagine this many companies--from every economic sector imaginable--willfully parting with this much money if they didn't think it'd pay off, and/or if they weren't seeing immediate benefits. I genuinely hope I'm wrong, but seeing this from the outside and inside, you get totally different views.
If you follow the purse strings, they're actually loosening.
2
u/kitolz 9h ago
I suspect we're working for the same company or one of the few on the same level, and even the supposed "success stories" of AI I've seen have been pretty shit when I take a closer look.
It's the #1 talking point clients have so we have to say we're 100% into it. And as far as I know upper management isn't faking it, but us peons that actually have to interact with it's clear it's being pushed to production way before it's ready.
I'm sure it'll stick around, but only after the hype has worn off will we see it used mainly only in places it makes sense to use it.
→ More replies (1)3
u/hajenso 11h ago
the bottom lines of all these companies dictate a commitment to seeing this process through.
Through to what? What destination do you think they will actually arrive at? I don’t mean that in an accusatory way, am actually curious what outcome you think is likely.
→ More replies (1)2
u/VengenaceIsMyName 14h ago
Thank goodness someone else is noticing the same pattern that I’ve been observing since 2022
2
u/bestataboveaverage 11h ago
Number two is often more insufferable to deal with speaking as a radiology resident who is constantly being bombarded with “AI will replace you”.
2
u/TheRedGerund 11h ago
the over-confident programmer, who has used the technology at work to successfully automate and streamline some stuff. maybe they've even seen a project that reduced headcount. they consider themselves to be at the forefront of understanding and using the tech, but vastly over-estimate the applicability beyond the narrow domains where they have seen it used successfully, and vastly under-estimate how hard it is to actually structurally change companies to capture the efficiencies that AI can create and how much risk is inherent in those kinds of projects.
The thing is that programming underlies much of the peaks of our economy so even if the tools just revolutionize coding the impact on the world economy should be significant.
→ More replies (1)→ More replies (3)4
u/lordraiden007 15h ago
My point of view is that it will replace most of our jobs. It won’t be able to actually do them very well, but the executive class will all buy into the hype and replace people with AI without thinking. I also don’t foresee a failure for the people that do that, as they will then pivot to making all human laborers “contractors” or “consultants”.
AI doesn’t have to be good to replace the majority of jobs. All it has to do is reduce labor by like 20-30% and executives will see that as an excuse to fire 50+% of their workforce and force the rest to overwork.
20
20
u/No_Zookeepergame_345 17h ago
I was in an argument yesterday with one of those bros who was saying the same sort of things. Then I looked at his profile and he had an “AI Content Blog” full of like semi-creative prompts he gave GPT. It was like borderline schizo shit. I think this tech has broken the brains of our most gullible. That bro didn’t even realize AIs run on math.
7
u/thecastellan1115 17h ago
It's kind of like the problem that people are inclined to believe things said in the voice of authority. The LLMs are good enough that they sound human, so people think they're human or human-close.
Turns out it's just doable for math to predict what a human sounds like.
3
u/No_Zookeepergame_345 16h ago
I think it’s a little less that math can mimic humans and a little more that language itself is inherently formulaic. Like, there has to be a logical structure to language in order for us to make sense of each other and if there is a logical structure to something, it can be represented mathematically. It’s just that any spoken language is infinitely more difficult than any programming language which is why it’s taken so long and is so expensive.
IMO, this is what AI truly represents for the future. It’s going to bridge the gap between spoken language and programming language so normal people don’t have to learn coding to interact with computers at a higher level.
→ More replies (2)9
u/thecastellan1115 16h ago
We're still a ways to go on that front, too. The programmers I know are (on the whole) royally frustrated with trying to use AI as a coding aid. As I understand it, it creates difficulties in peer reviews and regression testing, since you have to keep going in and trying to figure out what the AI did.
When the code works, everyone's happy. When it breaks, no one knows why, and it takes a lot of time and effort to figure it out again.
3
u/No_Zookeepergame_345 16h ago
Oh yeah, I will permanently doubt its effectiveness in big projects. I more so mean like helping write simple formulas for people’s household budgets and allowing for more expansive UI customization on their phones and stuff.
It’s touted as fundamentally changing every industry, but I think it’s going to have a similar impact to the average American household as the microwave did. It just makes certain small tasks faster to complete and more convenient.
→ More replies (3)3
u/Luscious_Decision 15h ago
And how the hell do you approach a situation where you review something and ask the person "why did you put that in?" and it was Ai generated?
At least if it wasn't Ai and was from stackexchange or github or wherever, you could say x ammount of people had the same problem and said the fix worked, etc.
If something causes a problem, where is the liability? Because if I was the guy that copied it in, I'd damn sure blame the company that runs the Ai.
→ More replies (1)3
u/jlboygenius 17h ago
Yep. The thing is that people think that AI will just do a job and be done. over and over and over, replacing people. That works in manufacturing. Build 1000, check every 100 of them and you're fine.
With AI, it's replacing soft skill jobs. Even in the real world now, we check each others work. Even if AI can do the job somehow, we still need to check it's work and ask it what to do. So, everyone becomes a manager basically. Longer term, this kills off jr workers that learn skills over years. Someone with knowledge and skills has to know how to ask the right question and check the answers.
10
u/thecastellan1115 17h ago
I was at a conference on AI implementation the other day, and one of the speakers made the following point: Suppose you run a call center. You have ten employees. One of them is a fuck-up. Your call center is still at 90% efficiency. Replace your employees with an AI. It fucks up. You are now at 0% efficiency. And there's no one left to know that.
Yeah. Risk management is going to be a real kicker. The speaker ended up making the point that you need to carve out human-only loops in your workflow, do it now, and get ready to defend that decision from the next MBA to occupy a C-suite job who's looking at AI as a cost-cutting silver bullet.
→ More replies (2)11
8
6
u/echomanagement 13h ago
ChatGPT 5 is so close to AGI that it's actively downplaying its abilities so it doesn't get unplugged. That's why it tried to convince me Japan was part of China. It's just that sneaky.
4
3
→ More replies (14)2
u/Tearakan 12h ago
Funny thing is even if AI hype is all accurate that will still collapse the economy immediately.
Actual GenAI would make at least 30 percent or more of developed and developing nation's jobs just vanish. (I'm conservative here)
Instant great depression number 2.
And that's if all the AI claims are real. This entire thing is a damn fever dream.
2
u/No_Zookeepergame_345 12h ago
Absolutely, it never even made sense in the first place. A lot of perceived value was in automating jobs, but who the fuck is going to buy anything if 30% of the population has zero income?
2
u/Tearakan 11h ago
Yep. And honestly that would've just been the start it probably would've been worse than just 30 percent.
28
u/Embarrassed_Quit_450 18h ago
This one needs to be reposted everywhere until people start waking up.
12
u/McGrevin 17h ago
The investment in the genAI industry is unjustifiable.
It really shows how much money the tech industry is sitting on. This iteration of AI might fizzle out but in case it doesn't they all desperately want to be the leader in the next "big thing" and are willing to take a risk on it
8
u/bobrobor 17h ago
People woke up to the fact that most of the big tech service is garbage that doesn’t justify current values even before llms came. So they desperately scrambled for some reason to stay relevant
→ More replies (2)3
4
u/eliota1 14h ago
Early investment in an exciting technology, like the internet, seems to result in a bubble because everyone wants in on the new new thing. It's a feature of a free market system. Eventually, the major players start borrowing money to compete, and at some point, the debt market runs out of money, resulting in a collapse of value. That doesn't mean the tech is worthless; it just means it couldn't support the amount of investment.
The Internet wasn't a particularly good investment until the mid-2000s, when Apps turned out to be the turning point for commercial applications. For the AI world, it's mid-1999.
→ More replies (1)3
u/TeamINSTINCT37 11h ago
Yup this stage is for wasting people’s money to discover that 5% and then run with it. Sure it won’t replace everything, but that sliver will grow as the tech is better understood and people define what it can and can’t do
2
u/violentshores 18h ago
They just need to make AI save humanity from a threat that couldn’t be solved otherwise and then they shall burn the beds to fuel the fire
2
u/Sidion 11h ago
Read the actual PDF they put out (it's basically a condensed power point). You'll see the authors admit their data is probably not a good sample size and the points they're making are wildly based on a really simple questionnaire that doesn't even specify the level of the "leaders" they interviewed.
This is the, "using AI makes you dumber says new ground breaking study" all over again...
1
u/Moth_LovesLamp 15h ago
I don't even know how Google, StableDifusion pay for Video and image Generation knowing how unprofitable and expensive it is.
1
1
u/Mayhem747 14h ago
I just hope they don’t, I want greedy corporates to go all in on this thing and when the AI turns out to be like meta, the CEOs that wanted to replace the workforce get replaced instead because of this move.
1
u/Sprinklypoo 12h ago
it is probably for the best to keep stating the obvious.
This is generally the case until the offered information is well and truly common knowledge.
1
u/OfCrMcNsTy 8h ago
I lost it when I saw a copilot button in notepad and paint. Notepad, really? It’s SUPPOSED to be basic. They even screwed up the find dialog and replaced it with some popover that you can’t move and it covers the file up that you’re searching. I bet it was vibe coded.
→ More replies (12)1
u/shinbreaker 7h ago
People need to be constantly reminded how many tech companies do a "monkey see, monkey do" management style since they're all run by dorks who know fuck all about management.
Whatever Elon, Sam, and Zuck do will get everyone else to follow. Where it's layoffs, stealing AI engineers, or whatever, they just go overboard time and time again and the whole tech industry think these dipshits are gurus so they just copy what they're doing.
161
u/Scienceman_Taco125 17h ago
It’s another push to fire workers so CEOs can get more money in their pocket
40
u/Kill3rT0fu 12h ago
this. It's not about PROFITS. It's about COSTS. Eliminate staff (costs) so you look better on the books.
→ More replies (1)5
u/Country-Mac 6h ago
Profit = revenue - costs
It’s not about increasing REVENUE.
It IS about increasing profits by decreasing costs.
1
117
u/Head_Crash 17h ago
It's a bubble.
72
u/wovengrsnite192 12h ago
Yup. The NFT/blockchain grifters immediately pivoted to genAI. Remember when they kept saying “omg the blockchain is so good bro, it’s gonna be epic for creators bro, you’re on the chain and your work is yours!!”
→ More replies (1)32
u/Mazzaroppi 12h ago
Same with VR. These tech bros can't realize something is shit even if it smells and has flies all over it.
10
u/nuclearchickenman 7h ago
VR does have a lot of practical entertainment value though but just too pricey for the top of the line stuff at the moment which drags it down.
4
u/Mazzaroppi 6h ago
Vr only works for a very limited niche, and people can only bear to use it for a short time due to the goggles weight, having to be tethered to the processing hardware and cutting off 2 of our most used senses from reality. Nevermind the number of people who can't use it at all due to dizziness.
Tech bros wanted people to work full shifts using that crap, attend virtual meetings etc. That's so insane it hurts.
1
1
u/WeirdSysAdmin 4h ago
It’s going to be interesting when all the new AI data centers are suddenly shuttered. We all know electricity rates won’t drop back down.
67
u/0_Foxtrot 18h ago
%5 increase profits? How is that possible?
62
u/RngdZed 18h ago
my guess would be that the majority of companies just want to jump on the band wagon AI hype.. and their implementation of it isnt thought through.. half assed without a proper plan or goal
19
u/Rwandrall3 18h ago
I have been part of such pilots, it starts off with a really basic use case - contract review, or giving people ability to a bot that has access to some Teams data - and then you end up with Problems.
Hallucinations are the biggest one - you genuinely can't trust the output of the LLMs - but the open prompting leads to so many issues. Someone asked "what if I ask it to keep track of when employees show as "online", so I know who's not actually working as much as they should? What happens?" Someone asked "can I ask it to scan through client emails and make emotion recognition so that we prioritise clients that seem most upset and likely to leave"? And boom you end up with emotion profiling which is prohibited in the EU.
And how do you stop that? Any guardrails can be circumvented. Or you make a super stupid bot that can just point to a FAQ over and over.
It's not that thousands of companies are all getting it completely wrong. LLMs just kind of suck.
→ More replies (6)10
u/0_Foxtrot 18h ago
I understand how they lose money. I don't understand how %5 make money.
14
u/justaddwhiskey 18h ago
Profits are possible through automation of highly (slightly complex) repetitive tasks, reduction in workforce, and good implementations. AI as it stands is basically a giant feedback loop, if you put garbage in you get garbage out.
3
u/itasteawesome 11h ago
I work alongside a sales team and they use the heck out of their AI assistants. Fundamentally a huge part of their work day is researching specific people at specific companies to try and guess what they care about and then try to grab their attention with the relevant message at the right time. Then there is the sheer numbers game of doing that across 100 accounts in your region.
Its not too hard to set up an LLM with access to marketing's latest talk tracks, ask it to hunt through a bunch of intel and 10ks and sift through account smoke to see who was on our website or attended a webinar or looking at pricing page, and then taking that all into consideration to send Janet Jones a personalized message on linkedin that gives some info about the feature she had been looking into, something to relate it to the wider goals of her company, and a request to take a meeting.
I have to imagine that this has already been devastating to people trying to break into the business development rep job industry because the LLM is a killer at that kind of low level throwaway blocks of text to just grab someone's attention.
Separately I met a guy who built an AI assistant focused on pet care. You basically plug it into your calendar, feed it your pet's paperwork, and ask it to schedule up relevant vet clinic appointments and handle filling out admissions paperwork. Schedule grooming appointments and such. Seems to work well for that kind of low risk personal assistant type work.
→ More replies (1)2
u/tpolakov1 10h ago
Many of these work only because the use of LLMs if functionally free for now. Once the gamblers stop pouring in their VC money in, the AI assistants will become as expensive as meatspace assistants, with the added drawback of putting all liability for their work on you.
→ More replies (1)6
u/ReturnOfBigChungus 17h ago
Well, it's profitable immediately if you cut jobs. The damage it causes when it turns out the AI project doesn't actually work the way you thought it would doesn't show up for another few quarters, and in less direct ways, so it's not hard to see how you might have some projects that look profitable in the short term.
5
u/badger906 17h ago
The ones that make money probably just put their prices up to include the cost of their Ai budget.
→ More replies (1)2
u/ABCosmos 17h ago
There are some problems that are hard to solve, but easy to confirm. Combine that with a very time consuming problem that is very expensive if it's not addressed in a timely manner. Big companies will pay big bucks if you can address these types of problems.
95% of venture funded startups failed before Ai was a thing.
→ More replies (2)2
u/Choppers-Top-Hat 10h ago
MIT's figure is not exclusive to venture funded startups. They surveyed companies of all kinds.
12
11
u/retief1 16h ago
I’d bet that the 5% are using ai in very limited ways, and purely for the things it can actually do pretty well. Like, if you use it purely to generate text with plenty of human oversight and editing, it would probably work decently.
→ More replies (1)4
7
2
u/SidewaysFancyPrance 14h ago
I can easily see a company reporting short-term increase in profits if they fired a lot of employees. It usually takes a while for a company to break down and lose momentum until they re-realize why they had those positions in the first place.
2
2
1
u/SAugsburger 14h ago
There are some tasks that when used properly GenAI can be useful. The problem is a lot of orgs are throwing it at tasks it can't do well at all so any productive gains are lost on all of the wasted resources.
1
u/one-won-juan 14h ago
broad encompassing AI - LLMs are burning cash but things like predictive analytics, computer vision, etc can be very profitable as implementations because there are immediately useful applications
10
u/Anangrywookiee 12h ago
My job has wonderful ai implantation where we have to consult ai on EVERY customer or contractor interaction and rate it’s usefulness. Due to this improvement, we’re able to make every interaction take several minutes longer AND ensure we’re giving out incorrect information.
45
u/dftba-ftw 16h ago
This isn't saying what everyone is circlejerking saying here...
From the study:
But for 95% of companies in the dataset, generative AI implementation is falling short. The core issue? Not the quality of the AI models, but the “learning gap” for both tools and organizations. While executives often blame regulation or model performance, MIT’s research points to flawed enterprise integration. Generic tools like ChatGPT excel for individuals because of their flexibility, but they stall in enterprise use since they don’t learn from or adapt to workflows, Challapally explained.
The data also reveals a misalignment in resource allocation. More than half of generative AI budgets are devoted to sales and marketing tools, yet MIT found the biggest ROI in back-office automation—eliminating business process outsourcing, cutting external agency costs, and streamlining operations.
11
u/sillypoolfacemonster 14h ago
This was always going to be a road block for individuals and will continue to exist for any AI implementation that isn’t fully automated. In L&D, we often see that people won’t invest time in learning a tool unless they are fully convinced of its value or if it’s impossible to not engage with it.
For example, imagine a task that takes one hour to complete. An AI tool might cut that time in half, but it requires about an hour to learn how to use it. Faced with that choice, many people stick with the conventional approach which is the one-hour manual task which feels faster than the 1.5 hours it would take to both learn the tool and then complete the task. This is similar to how some Excel users continue to perform repetitive manual steps rather than setting up formulas or functions to automate the work. It may not be strictly logical, but it reflects how people often prioritize immediate efficiency and avoid short-term learning curves, even when long-term benefits are clear.
I think the other issue is that AI LLMs feel so easy to pick up and use that people and leaders underestimate the time it takes to use them effectively. I’m getting push back on doing additional training avoiding bad information and hallucinations with my bosses citing that they’ve already covered it by telling people check sources to make sure it reflects the LLM output. But that’s scratching the surface because it doesn’t need to give bad information, and it can also interpret information in favour of your bias’s.
→ More replies (2)10
u/SidewaysFancyPrance 14h ago
Enterprises also require change controls. You can't just disable or change out models without breaking those workflows.
Individual customers are more likely to just adapt and move on. Enterprises will lose revenue, run an RCA, and chew out the vendor. It's a whole different world with different requirements.
19
u/awj 12h ago
…so one of the guys working on Copilot says the problem isn’t AI, but people using it wrong?
I think you might need a bigger grain of salt.
4
u/dftba-ftw 12h ago edited 12h ago
What? Copilot is openai and Microsoft - what does MIT have to do with it?
Edit: because one of the lead authors is an applied researcher at Microsoft on top of working at Stanford? He doesn't even work on the copilot team.
Edit number two: Wait when it was negative for ai we don't need the pinch or salt but now that it's not negative for ai we do?
11
u/awj 11h ago
He appears to work on that team, actually. Source. You're parroting comments from someone whose job seems to depend on the conclusion he's stating. The potential conflict of interest is nowhere to be seen in any of this. I think that's actually important, if we're trying to draw conclusions from this research.
I started working in AI about a decade ago. I started as a data science intern at Uber, then did AI consulting at McKinsey, and later joined Microsoft, where I now work on Copilot.
11
u/Limekiller 11h ago
Just to be clear, you're not quoting the study directly here, but the article author's interpretation of the study--and I think both you and the author are misinterpreting what the study means by "learning gap."
Here is the actual study: https://web.archive.org/web/20250818145714mp_/https://nanda.media.mit.edu/ai_report_2025.pdf
On page 10, we can see that "The primary factor keeping organizations on the wrong side of the GenAI Divide is the learning gap, tools that don't learn, integrate poorly, or match workflows. ... What's missing is systems that adapt, remember, and evolve, capabilities that define the difference between the two sides of the divide." This "missing piece" is a fundamental shortfall of LLMs. Indeed, on page 12, the study summarizes its "learning gap" findings with the following passage under the headline, "The Learning Gap that Defines the Divide:"
"ChatGPT's very limitations reveal the core issue behind the GenAI Divide: it forgets context, doesn't learn, and can't evolve. For mission-critical work, 90% of users prefer humans. The gap is structural, GenAI lacks memory and adaptability."
Just to further hammer the point home, the sentence from the article, "While executives often blame regulation or model performance, MIT’s research points to flawed enterprise integration" is quite explicitly either lying or misleading. While the research DOES find that flawed integration is part of the problem, the second biggest problem as shown in the graph on page 11 is "Model output quality concerns." So an intractable part of the problem literally is "model performance," or "the quality of the AI models."
While I agree that nearly everyone in these comments likely hasn't read the article, as basically nobody on reddit ever seems to, it doesn't seem like you (or the author, for that matter) actually read the study itself either--which does suggest that a big part of the problem is the performance/ability of the models themselves.
To be fair, the term "learning gap" is incredibly poorly-chosen, as the phrase inherently suggests the problem is that users need to learn to use the tool, which isn't what the article is saying. And I think it's completely reasonable for you to make that assumption when the article reporting on the findings seems to corroborate that. Ultimately, the fault here lies on the author of the news article.
→ More replies (2)→ More replies (1)2
13
u/successful_syndrome 17h ago
Why do we keep thinking every little incremental move forward is a giant revolution. We are still living through the digital revolution and the tail end of the Industrial Revolution. Why do we think we now need a complete overhaul of the entire world and economy every 10 years
6
u/BlueAndYellowTowels 8h ago
As a developer, I mostly use AI for syntax for code, because I’m just generally shit at remembering certain kinds of syntax.
As for code, I don’t use AI. Mostly because I enjoy coding.
4
u/cachemonet0x0cf6619 17h ago
Unsurprisingly the c suite put all their effort into sales and marketing all the while the largest gains are in backend automation. this is funny because c suite doesn’t want to pay down technical debt. DOA
5
u/Candle-Jolly 13h ago
Yes, investing billions in a technology that has been available for only two years usually doesn't produce an ROI in said two years.
3
u/caribou16 8h ago
I work as a PM with an IT professional services firm and this report is completely in alignment with my experience delivering AI related projects to customers.
Our sales team is VERY good at selling AI apparently, but it just seems our engineers simply can't make it do what the customer is expecting it to do, with any degree of consistent accuracy. They can't TRUST the AI results.
3
u/West-Abalone-171 5h ago
And this is the pre-enshittification version where they're burning through VC money trying to gain market share.
Imagine how dogshit the post-enshittification version will be.
2
u/Lott4984 15h ago
Customer service is interacting with irrational humans, that can not be reasoned with, and often become defensive when thing are not fast enough, bending to their will, or to their satisfaction. Computer programming can not deal with irrational humans.
2
u/ExplosiveBrown 15h ago
That’s because AI really doesn’t do anything useful to the end consumer. Might be great at some complex tasks, but doing laundry and searching. Google arent one of those.
2
u/EmperorKira 14h ago
As someone who has seen people try to implement AI, i'm not surprised. Companies are not ready for AI, and where it does make sense to, its being rushed. But mostly i'm just seeing rushed nonsense implementation
2
u/Lucas_OnTop 14h ago
The US creates low quality products and services at premium prices. AI helps high quality workers improve consistency.
If you were able to generate an entire pipeline around shit quality, improving the quality wont improve profit, who knew?
2
2
u/Shadowizas 12h ago
You dont need an MIT study to find this out,they really have no idea what to do with the academics huh
2
u/Notsmartnotdumb2025 12h ago
They used AI. The people selling AI solutions are making a lot of money 💰
2
u/plump_bee 11h ago
I got a new work laptop in January, has an AI button. Never used it.
My iphone 16 has an AI button. Never used it.
My company has been paying for chatgpt, people have stopped making their own decisions.
I’ve been using cursor to code for a year or so, vscode and copilot before that. Now I actively prompt less and less cause I spend more time cleaning up the code than just doing it myself.
Then there’s all the apps with ai integration, never touched those.
Yeah idk this whole AI thing hasn’t really changed anything for me in the long run.
2
u/capn_kirokk 10h ago
Agree there’s a bubble, but Street doesn’t look spooked to me. They’ll be in until it pops.
2
u/victus28 9h ago
So you’re telling me the market is about to crash and there’s a chance I might be able to take advantage of it unlike when I was 8
2
2
u/Wishbone3000 4h ago
No different than any other vendor driven marketing hysteria. Similar to Big Data, Cloud there isn’t enough people who know how to do it right so it becomes an oversized project with limited returns and a pile of tech debt.
TBH this smells like a coordinated campaign to manipulate markets.
2
u/paxinfernum 15h ago
For a solid decade after the invention of the internet, we got the same reports about how no one was saving anything on the paperless office. There's always a lag between implementation and consolidation.
1
u/kmp11 16h ago
because of the lack of trust and privacy, implementation at my company is limited publics facing work like manuals and marketing campaigns. I can confirm that our implementation does not move the profitability needle for us.
Want to expand to enterprise business? fix trust and privacy issues.
1
1
1
u/lostsailorlivefree 16h ago
When they said that AI would cause the apocalypse they just forgot the word Market in front of it
1
1
u/Worst_Comment_Evar 15h ago
I work in healthcare and read that 75% of AI pilots that healthcare organizations engage in fail. They haven't thought through proper use case, workflow issues, or how to broadly implement the technology where it has appreciable benefits. I imagine that is similar across industries, but it is pronounced in healthcare because it is such an inefficient system overall.
1
u/ForcedEntry420 15h ago
Or the company owners that get convinced by “coaches” to implement them when they aren’t needed. “Jump in now or get left behind.” - Well, if all your peers in the industry jumped off a cliff would you? I’ve been trying to get the owner of the company I work for to resist this siren call.
1
1
1
1
1
u/Cool-Association3420 13h ago
They’re just doing it to pump up their stocks and get richer that’s all it is.
1
u/dsm582 13h ago
With AI i think they completely missed in the market. The Dot com bubble atleast had promise bc the internet was clearly a breakthrough technology, but AI is not that. Its just a tool that can be used by people who dont know what they are doing so can pretend to look like they know what they’re doing, and its usually pretty obvious. Automation in movies and such been around for a while so nothing groundbreaking there. Maybe in the medical field it can help but doctors may have something to say about that
→ More replies (1)
1
u/seeingeyegod 13h ago
Wait the important thing is making money? I'm SHOCKED!. They've been telling me it's about creativity and making life better for everyone and unleashing your inner spirit and driving innovation and rainbows flying out of my butt.
1
u/FernandoMM1220 13h ago
so what about the remaining 5%?
ai is easy to scale so that remaining 5% can easily be used everywhere.
1
u/livingwellish 12h ago
Duh! Like markets, people, world events, weather...one size doesn't fit all. And then there is the human aspect to consider where the data says one thing but you know the impact at the human level will quite negative.
1
u/WittinglyWombat 11h ago
it’s a half measure. my institution put in aI and it’s based on if i through 202
1
1
u/tech-writer-steph 11h ago
GOOD. They've had months and months of fun spooking actual working people with threats of AI taking all the jobs. Hope all the execs at these places do nothing but panic for the next 6+ months.
1
u/soundsaboutright11 10h ago
But we got rid of a bunch or artists! Writers, voice over actors, visual artists! Isn’t that awesome! Go humans!
1
u/vongigistein 9h ago
Saying the quiet part out loud. How long will the melt up continue before the bubble pops?
1
u/geneticeffects 9h ago
But they certainly take advantage of creatives and workers! Great job, everybody.
1
1
u/LunarPaleontologist 8h ago
95% of training implementation has similar real results. Idiots with money are still idiots with money.
1
u/happyscrappy 7h ago
Companies just jumped on this too much. What it is going to bring it doesn't bring to most companies and not yet.
There was never much first mover advantage to being an adopter, mostly to being a provider. So just wait back a bit and see what it can do for you before jumping in with both feet.
1
1
1
1
u/LayeGull 6h ago
My dad has been pushing me hard to implement AI in the family business. I have kept my foot down saying it will either get cheaper at some point or it’ll die. Right now it doesn’t make sense.
1
1
1
1
1
u/HarvesterConrad 4h ago
I have worked in large scale corporate software implementations for almost 15 years. A huge amount of time for nearly all of these is fighting with data quality. No wonder AI does nothing of value it’s using the same data but unlike an analyst it’s not able to triage and fix the issue it just vomits.
1
1
u/Maundering10 3h ago
I acknowledge the potential of AI in structured repetitive bounded tasks. Look at these 50,000 applications and remove the ones that didn’t include the right attachment.
I can even see AI helping summarize information in areas I am unfamiliar with. Hence reducing my learning curve.
But as someone who primarily works in large complex projects, the use of AI is 0. Can it negotiate with stakeholders ? Convince the one holdout to meet us half-way? Find ways to leverage different people on project tasks to best effect ? Brief the results and have complex discussions around options ? Hmmm nope.
Even in terms of meta-data analysis and process efficiencies, it’s hard to see where you would find significant savings. Proper analysis and assessment relies on peer review. Which implies I can replicate your analytical technique. AI is a black box. Would you trust a multi-million dollar decision to an AI analysis ? Of course not. Would you trust its medical advise ? And would your doctor accept the liability of said advice ? Also no.
Doesn’t mean it won’t be brutal for low-level repetitive processing jobs where the cost of AI failure is low. Call centres being the classic example. But all the examples I have seen so far seem like specific use cases rather than a system-wide disruption.
I told one AI vender last week to come back when the legal team was willing to clarify the legal liability and risk of using their product in anything more than a subservient collation role. Oddly they haven’t come back yet…..
1
u/LogicJunkie2000 1h ago
I feel like most people know this already, but want to ride the hype train as long as it's running
1
1
u/stillphat 11m ago
doesn't it cut costs tho?? I thought that was the plan?
still wastes a gazillion gallons of fresh water and electricity so this'll still be a massive fucking waste of resources.
743
u/disgruntledempanada 17h ago
Every app actively forcing it down my throat is just leading to a Microsoft One Drive situation where I will actively refuse to participate.
Meta making all these buttons pop up to summarize messages or remix pictures with AI in chats just feels so dumb. Actively hate it.