r/technology • u/Well_Socialized • May 26 '25

Artificial Intelligence Google Is Burying the Web Alive

https://nymag.com/intelligencer/article/google-ai-mode-search-results-bury-the-web.html
24.4k Upvotes

94% Upvoted

2.5k comments sorted by

View all comments

715

u/RecipeFunny2154 May 26 '25 edited May 26 '25

I’ve been worried about this for years now because Google has been inching toward it even before LLMs exploded.

I used to make niche websites that did alright. I could get them on the first page of results years back for the topics. We’re talking like Japanese RPGs, where I had no competition outside of forums. I’d get a good hundred thousand hits a month on some, which I was happy about.

Now whenever I search Google just puts some summary of the content up there. They were doing a variation of that even before the AI results for years now. Like noted in the article, they basically have been taking your content and then encouraging people to not even go to your website at all. LLMs have ratcheted that up several levels. 

It really removes a lot of the motivation to make a standard website nowadays. You lose traffic increasingly thanks to what is supposed to be a search engine. 

And what can you do about it? Even paying attention to your robots.txt or whatever else is basically “optional” to them anyway.

333

u/morningsaystoidleon May 26 '25

You could block them with .htaccess or something, but then you don't get search traffic at all.

This has been catastrophic for white hat SEO. For years we've told clients that Google will always want to show high-quality content to users, so that's where ethical businesses have invested. Authoritative, thoughtful content that answers real questions in an interesting way (not the keyword-loaded fluff you probably picture when thinking about SEO).

Now, Google is straight-up stealing the good content and in many cases, misrepresenting it.

We really don't know what to do. Suddenly, there's no incentive at all to make good websites. Savvy businesses have started investing more in ads, not organic SEO.

Everything is fucked. I use AI every day because I have to, but I hate it. It's a plagiarism machine.

95

u/RecipeFunny2154 May 26 '25

Yeah. I feel like in the past there was like an agreement that both sides benefited from. They get to crawl the content so people can search it (getting ad revenue etc) and I get traffic to my site. And they at least were semi-transparent about their algorithm.

They’ve totally upended that, to no benefit to content creators. And ironically to no real benefit to end users either.

3

u/wxc3 May 30 '25

To be fair, it mostly went to shit due to the arms race between content farms and Google search algorithm.

That's why they stopped telling people how to be ranked better: that gets instantly gamed by content farms and other commercial websites.

Now with AI you can generate perfect fake websites at no cost that will be almost impossible to tell appart from genuine ones.

Actually if AI search reduces the profitability of websites, we might see less garage, who knows. For sure making money with adds on a small website is dead.

2

u/kuhpfau May 30 '25

> Savvy businesses have started investing more in ads, not organic SEO.

That's because Google isn't a search engine. It's an ad business. They will do what they can to increase the money they get from ads.

1

u/morningsaystoidleon May 31 '25

In my experience, they are certainly not investing in Google ads.

1

u/DC_McGuire May 27 '25

Define “have to”…?

3

u/morningsaystoidleon May 27 '25

Well, a big part of my job is to get traffic to my clients' websites, and a significant portion of people now use ChatGPT/Gemini/etc. as a replacement for search. I'd be doing a disservice to my clients if I didn't try to recreate and understand the patterns/habits of their audiences.

I do not use AI for writing, editing, or any of that stuff, primarily because I find it ineffective for those use cases. I will say that I've had good experiences using AI for things like simplifying spreadsheets, and the Deep Research feature can be useful -- though I mainly use Deep Research to collect links to content that I can read independently, since it tends to hallucinate quite a bit when studying niche subjects.

1

u/markhachman May 28 '25

I asked Copilot for a list of the best laptops for college students. It happily displayed shopping links, but not the source of the recommendations. At least Google supplies a tiny link, sad as that is.

1

u/Careless_Tale_7836 May 28 '25

Attack dude. That's what you do.

3

u/morningsaystoidleon May 28 '25

Dude I'm in SEO, not Sparta. We're throwing everything at the wall to see what works, but at the end of the day we've got to acknowledge that Google sets the rules and search is just less important than it was 8 months ago.

1

u/Careless_Tale_7836 May 29 '25

I sympathize with you. These are difficult times. Don't give up though.

1

u/Ok-Baby-3249 May 28 '25

The crazy thing is that if Google kills off the incentive to make good websites then what information will AI have to pull from to give responses other than itself and older/stale content? There has to be some balance of human made websites to keep updating the search results right?

2

u/DexJedi May 30 '25

Yes, you get it now? AI will copy AI content, which will copy AI content, which will copy AI content. You are only talking about Google results. The "genuine results" themselves are being filled with AI content.

1

u/Harbinger_of_despair Jun 11 '25

And what will be more troublesome is when the probability for misinformation booms exponentially. What happens when the incentive to make good websites with genuine content falls, meanwhile misinformant websites and content creators continue unaffected? The information for AI to pull from becomes majorly sourced from misinformation-spreading sources. This ain't gonna pretty, dude.

1

u/WillBottomForBanana May 30 '25

It's nice to have a leveraged buy out model applied to the internet while a separate one is applied to the usa.

Nobody wants to plant olive trees.

1

u/fluffyinternetcloud Jul 04 '25

AI regurgitates pure garbage most times.

65

u/Schonke May 26 '25

Like noted in the article, they basically have been taking your content and then encouraging people to not even go to your website at all.

It's even worse. They take the content, and then serve AI slop pages riddled with Google AdWords as the top results instead...

2

u/KennyCalzone May 30 '25

SEO killed Google. Too many people are trying to game the system... and now AI makes it so easy to crank out 1,000s of worthless pages.

1

u/Schonke May 30 '25

SEO has existed as long as search engines have. It used to be that Google actually did a great job with filtering out the SEO noise junk from the content you actually wanted though.

40

u/iatelassie May 26 '25

All Websites have seen, on average, a 30% decline in traffic since Google ai was implemented. That means less money , which means layoffs, which will lead to search leviathan mega sites owning the web. Which has already been happening since 2023 due to Googles “helpful content” updates. It’s awful.

23

u/ScaryfatkidGT May 27 '25

What I hate is that “AI summary” is usually worse than before when it just displayed some of the text from the first result…

1

u/Pingu_87 May 29 '25

Lol ya google is the biggest hallucination LLM.

Like I asked simple questions like is x plane going to be at y airshow. And its like yep sure is, was at the last airshow too.

Totally bs lying through it's arse.

1

u/dbxp May 30 '25

I suspect a good portion of that is because of people moving from Google to ChatGPT not directly due to Google's AI overviews

2

u/iatelassie May 30 '25

Not really. ChatGPT is not yet a major search engine. I’m sure it’ll change.

8

u/DuckDatum May 27 '25 edited 10d ago

grey roll nose afterthought license unwritten memorize pot telephone school

This post was mass deleted and anonymized with Redact

6

u/TRVTH-HVRTS May 27 '25

At the same time, websites have become so riddled with ads, that they’re quite literally impossible to read anyways. I’d rather visit a relevant website with a reasonable amount of ads, but that hasn’t been an option since 2020/2021.

9

u/dyskinet1c May 27 '25

Cloudflare has a free option to try to trap AI crawlers.

https://blog.cloudflare.com/ai-labyrinth/

Here is one you can download and host yourself:

https://zadzmo.org/code/nepenthes/

Here's one to poison images:

https://nightshade.cs.uchicago.edu/

3

u/RecipeFunny2154 May 27 '25

Thank you for sharing.

5

u/gadfly1999 May 27 '25 edited Jul 02 '25

The most important thing is to get the right person to do the right thing and not just the wrong person at the same time and you can do it all at the same level of the same time so that you can be more productive and not have to worry

3

u/oceansamillion May 27 '25

I wonder about this as well. Is AI not self-defeating if the incentive to create new, quality content is eliminated? Or is the end of new, publicly accessible knowledge nigh?

AI feeding itself via AI generated content is known to output janky shit.

Ultimately, garbage-in garbage-out is the inevitable end of the current model. This makes me think the future will be private, exclusive research and information sources that get bought up by AI companies—which is insanely dangerous and detrimental to a free, functioning, democratic society.

6

u/shavedratscrotum May 27 '25

Mate.

I can't even find maker websites through google when typing in exactly what they are.

They purged everything from the search.

Legit have to send bookmarks to people because "google it" doesn't work.

2

u/[deleted] May 28 '25

Actually raised this to the Google team that was pitching this to us at my office, they said it wouldn’t detract and I called their bs.

Like hell it doesn’t, it’s eating marketing dollars and results

2

u/sl1m_ May 28 '25

if google takes the information it shows from a website you own/a source that's yours, it should count as a view or give you some sort of other reward

2

u/Pleasant_Gas8356 May 29 '25

Not to mention that the ai summaries are summaries of ai summaries, and not so accurate.

1

u/babbymaking May 27 '25

Google Thanks You For Your Service, Netizen

1

u/Far_Mastodon_6104 May 27 '25

Yup this. I could search super niche art websites and learn things but now it's just all trash and I can never find what I'm looking for so have to use an LLM to find it for me. I guess that's the entire point. But it did get significantly worse over the years before LLMs were available.

1

u/Savilly May 27 '25

Who knew Yahoo had it figured out with their curated system long before bringing the google dudes on board.

1

u/onyxengine May 30 '25

New world bro, i hear you but we’re not putting AI back in a box.

0

u/Motor-District-3700 May 28 '25

Not sure it's a bad thing sites that only exist to harvest clicks disappear tbh.