Bot Gaffe
Meta's AI Crawlers Nearly Took an Artist's Business Offline
Meta's web crawlers visited San Diego children's author Chelsea Paulus's website more than 550,000 times in 48 hours, pushing her hosting CPU usage to 80%, then 100% when a second Meta bot arrived. She edited robots.txt and got her host to add blocking code, but still had to remove the webpage link from her Instagram profile to keep human clicks from finishing the site off. Meta says site owners can manage its crawlers through robots.txt.
Source: Business Insider