533
Google’s AI Is Destroying Search, the Internet, and Your Brain
(www.404media.co)
you are viewing a single comment's thread
view the rest of the comments
view the rest of the comments
this post was submitted on 23 Sep 2026
533 points (97.3% liked)
Technology
88648 readers
4053 users here now
This is a most excellent place for technology news and articles.
Our Rules
- Follow the lemmy.world rules.
- Only tech related news or articles.
- Be excellent to each other!
- Mod approved content bots can post up to 10 articles per day.
- Threads asking for personal tech support may be deleted.
- Politics threads may be removed.
- No memes allowed as posts, OK to post as comments.
- Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
- Check for duplicates before posting, duplicates may be removed
- Accounts 7 days and younger will have their posts automatically removed.
Approved Bots
founded 3 years ago
MODERATORS
I think it’s cute you think the AI sites are paying attention to robots.txt. 🤣
I don’t; I think googlebot is paying attention to it, and AI sites get a LOT of their initial visibility and site ranking from the Google index.
Case in point: the sites I spend time on that set robots.txt to deny all bots before the LLM wave started still aren’t getting hammered with AI bot requests, while the ones that didn’t are.
I can see it on my own websites as well; I’ve got one that’s in Google’s index, and it gets a constant low volume traffic from AI bots. The ones that aren’t listed? They just get traffic from exploit scanners.