I've been hit many times over as well (smallish forum website with about 12000 posts).

Seen it all: fake user agent strings, ignoring robots.txt, either localized IPs (lots from China) or distributed, load increasing to 500 times the normal value, until the site goes down.

For now, a combination of these keeps it manageable:
- fail2ban
- apache mod_evasive
- restricting forum access to logged in users

When the forum is accessed by a crawler, they get a short paragraph about how great the site is, generated by ChatGPT :-)

Can we have a subdomain tiobe . slashdot . org so the monthly promotion post can be parked there?

why do I have to go through MSN :-/

link to the Atlantic

It's the basic recipe for publicity now:

state that <difficult task> will be done <high percentage> by AI in <only a few> years, and boom, people/news sites will bite

the field is so crowded now, and this is one of the easiest ways to get attention

Exactly, they use fake user agents, and ignore robots.txt

I run a small Calcudoku puzzle site, and recently had to take the forum offline because they were hitting it so hard the server load went up to 300+ (a normal high load is 3..).

For the past few months the crawlers are all from Chinese IPs.

I'm looking to create custom fail2ban rules to detect and block them.

you mean MRGA

I'll happily pay a monthly fee if this means I can use iplayer and sounds

Not sure why the discussion is complicated.

Everybody knows the FIDE World Champion is not the strongest chess player in the world.

The mechanism is strange too.

Imagine in tennis, for example, one player who is "World Champion" goes downhill and slides to #25 in the world.

Then there is a tournament with 8 of the top players, and the one ranked #5 wins it.

Then at the end of the year there's a tournament with many matches between the #5 and #25 (so might as well cancel the ATP Tour finals), and the winner of that gets to call himself World Champion.

Seems like a good system, this should be introduced in every sport.

That's my conclusion by now as well.
On and off I posted about how I can't understand why this is "news" _every month_

Now my guess is that Tiobe pays slashdot somehow.

time to rewrite systemd in Rust

pros? cons?

slashdot needs AI to detect the dupes.

the only risk is it'd put the editors out of a job.

non-paywalled link:

https://interestingengineering...

This one interesting too, from Simon Prins his blog:

https://www.simonprins.com/dog...

Why is there a TIOBE post only once a month?

They should switch to a weekly story.

Amen to that. I still don't get why this is news just about every single month.

I'm guessing slashdot has a deal with Tiobe, so they get a cut of the ad revenue.

Isn't there a 1000 word limit for a Slashdot summary?