Overcoming anti-scraping protection
Getting blocked often? Use full blown headless Chrome, intelligent proxy and many more...
+1
6 articles in this collection
Written by Jan Čurn, Lukáš Křivka, Jaroslav Hejlek and 1 other
Written by Jan Čurn, Lukáš Křivka, Jaroslav Hejlek and 1 other
Free Trial of Apify Proxy
Try advanced IP address rotation in your crawlers and actors free of charge
Written by Jan Čurn
Several tips how to bypass website anti-scraping protections
Learn how to scrape data from websites that employ IP rate limiting, browser fingerprinting and other modern bot detection techniques
Written by Jaroslav Hejlek
Using stealth mode to mask headless Chrome
Overcoming anti-scraping protection with browser fingerprinting-proof stealth mode
Written by Petr Pátek
How to handle blocked requests in PuppeteerCrawler
Getting around website defense mechanisms when crawling
Written by Lukáš Křivka
Filter out blocked proxies using sessions
Handling blocked requests correctly
Written by Lukáš Křivka
Using man-in-the-middle proxy to intercept requests in Puppeteer
This article demonstrates how to setup a reliable interception of HTTP requests in headless Chrome / Puppeteer using a local proxy.
Written by Jaroslav Hejlek