> This increase is not coming from human readers, but largely from automated programs that scrape the Wikimedia Commons image catalog of openly licensed images to feed images to AI models.
Sounds like the problem is not the crawling itself but downloading multimedia files.
The article also explains that these requests are much more likely to request resources that aren't cached, so they generate more expensive traffic.
Sounds like the problem is not the crawling itself but downloading multimedia files.
The article also explains that these requests are much more likely to request resources that aren't cached, so they generate more expensive traffic.