They serve a different robots.txt to Google: https://merj.com/blog/investigating...

		sunaookami on July 24, 2024 \| parent \| context \| favorite \| on: Google is the only search engine that works on Red... They serve a different robots.txt to Google: https://merj.com/blog/investigating-reddits-robots-txt-cloak... You can see it here: https://search.google.com/test/rich-results/result?id=_mYogl... (click on "View Tested Page")