Overcoming anti-scraping protection

Getting blocked often? Use full blown headless Chrome, intelligent proxy and many more...

+1

6 articles in this collection
Written by Jan Čurn, Lukáš Křivka, Jaroslav Hejlek and 1 other

Free Trial of Apify Proxy

Try advanced IP address rotation in your actors and tasks free of charge

Written by Jan Čurn
Updated over a week ago

Several tips on how to bypass website anti-scraping protections

Learn how to scrape data from websites that employ IP rate limiting, browser fingerprinting and other modern bot detection techniques

Written by Jaroslav Hejlek
Updated over a week ago

How to handle blocked requests in PuppeteerCrawler

Getting around website defense mechanisms when crawling

Written by Lukáš Křivka
Updated over a week ago

Using stealth mode to mask headless Chrome

Overcome anti-scraping techniques with browser fingerprinting-proof stealth mode when web scraping

Written by Petr Pátek
Updated over a week ago

Filter out blocked proxies using sessions

Handling blocked requests correctly

Written by Lukáš Křivka
Updated over a week ago

Using man-in-the-middle proxy to intercept requests in Puppeteer

This article demonstrates how to setup a reliable interception of HTTP requests in headless Chrome / Puppeteer using a local proxy.

Written by Jaroslav Hejlek
Updated over a week ago