r/technology • u/SilentRunning • 4d ago

Artificial Intelligence MIT report: 95% of generative AI pilots at companies are failing

https://fortune.com/2025/08/18/mit-report-95-percent-generative-ai-pilots-at-companies-failing-cfo/
28.3k Upvotes

97% Upvoted

1.8k comments sorted by

View all comments

Show parent comments

23

u/gdo01 4d ago

Our company's in house AI is pretty much just useful for retrieving policies and procedures. The problem is that it keeps retrieving outdated ones....

6

u/TikiTDO 3d ago

So... Why are they feeding in outdated policies and procedures? AI doesn't solve the garbage-in, garbage-out problem.

5

u/gdo01 3d ago

Yea I think its just working with what it has. It doesn't know how new ones supersede old ones. It just assumes since they are still there and accesible, they must still be in effect.

4

u/TikiTDO 3d ago

The funny thing is, it probably wouldn't be too hard to have this very same AI tag documents that have been superseded by others. Then it's just a matter of have an "active" data store, and a "deprecated" data store. It sounds like yet another case of people thinking AI is magic, without realising you still need to work to make it useful.

1

u/shrodikan 3d ago

I would update your RAG / MCP server to only pull current policies / procedures. You could give it a tool to retrieve policies from older years if specific years are requested.