AI crawlers run up a website's hosting bill

Read the Docs, a third-party documentation hosting service has found its bandwidth bill skyrocketing due to AI crawlers.

Web crawlers for search have long used a surprising amount of bandwidth and sometimes CPU resources. Website owners, however, want these crawls in most cases because they are how discovery now works on the Internet. — Read the rest

The post AI crawlers run up a website's hosting bill appeared first on Boing Boing.