Cloudflare is one of the most popular Content Delivery Networks (CDNs). Like most CDNs, Cloudflare also comes with optional anti-scraping features.

As with many other anti-scraping systems, there is high variability in the settings of each website that will set the blocking system to be more aggressive or "friendlier" towards bots. There isn't any silver bullet but using Apify's infrastructure with Crawlee and following a few core rules does generally overcome the protection.

Be especially mindful of the Cloudflare browser check and challenge, it might require some additional steps in your code (examples will be added soon).

You can learn more about bypassing Cloudflare in Web scraping: how to crawl without getting blocked.