Overcoming anti-scraping protection

Getting blocked often? Use full blown headless Chrome, intelligent proxy and many more...

+1

6 articles in this collection
Written by Jan Čurn, Lukáš Křivka, Jaroslav Hejlek and 1 other

Free Trial of Apify Proxy

Try advanced IP address rotation in your actors and tasks free of charge

Written by Jan Čurn

Several tips on how to bypass website anti-scraping protections

Learn how to scrape data from websites that employ IP rate limiting, browser fingerprinting and other modern bot detection techniques

Written by Jaroslav Hejlek

How to handle blocked requests in PuppeteerCrawler

Getting around website defense mechanisms when crawling

Written by Lukáš Křivka

Using stealth mode to mask headless Chrome

Overcome anti-scraping techniques with browser fingerprinting-proof stealth mode when web scraping

Written by Petr Pátek

Filter out blocked proxies using sessions

Handling blocked requests correctly

Written by Lukáš Křivka

Using man-in-the-middle proxy to intercept requests in Puppeteer

This article demonstrates how to setup a reliable interception of HTTP requests in headless Chrome / Puppeteer using a local proxy.

Written by Jaroslav Hejlek