All Collections
Developing Actors
How to bypass Cloudflare
How to bypass Cloudflare

Apify's default scrapers are a great choice for Cloudflare protected sites

Lukáš Křivka avatar
Written by Lukáš Křivka
Updated over a week ago

Cloudflare is one of the most popular Content Delivery Networks (CDNs). Like most CDNs, Cloudflare also comes with optional anti-scraping features.

As with many other anti-scraping systems, there is high variability in the settings of each website that will set the blocking system to be more aggressive or "friendlier" towards bots. There isn't any silver bullet but using Apify's infrastructure with Crawlee and following a few core rules does generally overcome the protection.

Be especially mindful of the Cloudflare browser check and challenge, it might require some additional steps in your code (examples will be added soon).

You can learn more about bypassing Cloudflare in Web scraping: how to crawl without getting blocked.

Did this answer your question?