Fediverse

36271 readers

186 users here now

A community to talk about the Fediverse and all it's related services using ActivityPub (Mastodon, Lemmy, KBin, etc).

If you wanted to get help with moderating your own community then head over to !moderators@lemmy.world!

Rules

Learn more at these websites: Join The Fediverse Wiki, Fediverse.info, Wikipedia Page, The Federation Info (Stats), FediDB (Stats), Sub Rehab (Reddit Migration)

founded 2 years ago

MODERATORS

280

submitted 6 months ago* (last edited 6 months ago) by mesamunefire@lemmy.world to c/fediverse@lemmy.world

52 comments fedilink hide all child comments

We have paused all crawling as of Feb 6th, 2025 until we implement robots.txt support. Stats will not update during this period.

you are viewing a single comment's thread
view the rest of the comments

[–] swizzlestick@lemmy.zip 28 points 6 months ago (3 children)

It would be better if the server could detect bots and send them down a rabbit hole

Already possible: Nepenthes.

[–] Semi_Hemi_Demigod@lemmy.world 26 points 6 months ago

ANY SITE THIS SOFTWARE IS APPLIED TO WILL LIKELY DISAPPEAR FROM ALL SEARCH RESULTS.

I’m sold

[–] Skepticpunk@lemmy.world 3 points 6 months ago

Ooh, nice.

[–] merthyr1831@lemmy.ml 2 points 6 months ago

that website feels like uncovering a piece of ancient alien weaponry