Overcoming anti-scraping protection
Getting blocked often? Use full blown headless Chrome, intelligent proxy and many more...
+1
6 articles in this collection
Written by Jan Čurn, Lukáš Křivka, Jaroslav Hejlek and 1 other
Written by Jan Čurn, Lukáš Křivka, Jaroslav Hejlek and 1 other
Free Trial of Apify Proxy
Try advanced IP address rotation in your actors and tasks free of charge
Written by Jan Čurn
Several tips on how to bypass website anti-scraping protections
Learn how to scrape data from websites that employ IP rate limiting, browser fingerprinting and other modern bot detection techniques
Written by Jaroslav Hejlek
How to handle blocked requests in PuppeteerCrawler
Getting around website defense mechanisms when crawling
Written by Lukáš Křivka
Using stealth mode to mask headless Chrome
Overcome anti-scraping techniques with browser fingerprinting-proof stealth mode when web scraping
Written by Petr Pátek
Filter out blocked proxies using sessions
Handling blocked requests correctly
Written by Lukáš Křivka
Using man-in-the-middle proxy to intercept requests in Puppeteer
This article demonstrates how to setup a reliable interception of HTTP requests in headless Chrome / Puppeteer using a local proxy.
Written by Jaroslav Hejlek