University libraries are being swarmed by AI bots, putting public access at risk

[ownedbycats]

https://www.cbc.ca/news/world/univer...bots-9.7408287

Eric Hellman, executive director of Project Gutenberg — a U.S.-based volunteer organization that digitizes and archives cultural works and distributes eBooks — wrote in a blog last year that these bots, which he characterizes as poorly programmed "monstrosities," are "destroying open access" on the internet.

He tells CBC News that "everything's gotten worse" since he wrote that.

Hellman says the problem comes down to good bots and bad bots.

He explains that search engines like Google, and archiving sites like Internet Archive, use bots that follow robots.txt — voluntary sets of instructions on websites that tell them what they can and can't do, so they don't disrupt the site's functioning while gathering information.

AI scraping bots, on the other hand, sometimes ignore these rules and overwhelm websites — in some cases requesting thousands of documents a second and downloading them in every available format. [...]

He says Project Gutenberg is happy for its materials to be used as AI training data and even offers a one-click download of its entire library of books, "but the bots are too stupid to use it."

[Gormagon]

The bots just do what some coders told them to do...or if LLMs were used the LLMs produced rubbish when bots aren't following the restrictions of the robots.txt. There's also the possibilty that the bots were designed the way they act, i.e., not to obey any restrictions on websites.

[DNSB]

Originally Posted by Gormagon

There's also the possibilty that the bots were designed the way they act, i.e., not to obey any restrictions on websites.

Personally, I would vote for the bots being designed that way. My honeypot has been hit by multiple bots that basically wanted to download the entire site including a bunch of relatively small zip files that were designed to expand to multiple terabytes of output files. Do a search on 'decompression bomb' or 'zip bomb' for more on these files. Most utilities these days will detect some of these but still a PITA to deal with.

[nana77]

The even worse (imho, heard it this evening on broadcast) is that all of those server infrastructures in construction (there are quite a bunch in here too) are going to use ALOT of water, too.

[Gormagon]

And the problem is? The water is still there or do these servers use electrolysis to produce H₂ and O₂?

[DNSB]

Originally Posted by Gormagon

And the problem is? The water is still there or do these servers use electrolysis to produce H₂ and O₂?

And after the water is used to cool the servers, is it dumped into a river, sewer, whatever or is it cycled through a cooler and re-used?

[j.p.s]

Originally Posted by Gormagon

And the problem is? The water is still there or do these servers use electrolysis to produce H₂ and O₂?

The water problem is that some of these centers are being built in areas that are already using water faster than it is being replenished, such as Arizona, USA. (And really any area using Colorado River water. This is just an obvious example, there are many areas that do not have extra water available.)

Just because the water still exists on Earth doesn not mean water use is not a problem.

Note that even the recirculating systems still lose water.