Extracting data

Use Apify actors or scrapers to extract any data you need from any website.

+4

13 articles in this collection
Written by Jan Čurn, Jakub Drobník, Marek Trunkát and 4 others

Waiting for dynamic content

You load the page. You execute the correct selectors. Everything should work. It doesn't? Learn how to wait for dynamic loading.

Written by Lukáš Křivka

Request labels and how to pass data to other requests

How to handle request labels in Apify actors with Cheerio or Puppeteer Crawler

Written by Václav Růt

Scraping a list of URLs from a Google Sheets document

Learn how to crawl a list of URLs specified in a Google Sheets document using one of the Apify web scraping actors.

Written by Jan Čurn

How to scrape pages with shadow DOM

Shadow DOM enables the isolation of web components, but causes problems for web scrapers. Here's an easy workaround.

Written by Marek Trunkát

Scraping iframes with Puppeteer

How to get information from inside iframes using Puppeteer

Written by Lukáš Křivka

When to use Puppeteer Scraper

Choosing between Web Scraper and Puppeteer Scraper can be difficult. We explain the important differences to help you pick the right tool.

Written by Ondra Urban

How to scrape a website using sitemap.xml and Apify SDK

Website with proper sitemap.xml = free jackpot for every web scraper

Written by Marek Trunkát

Submit form with file attachment using Puppeteer

How to submit a form with attachment using Puppeteer.

Written by Jan Čurn

Send multiple POST requests to the same URL

Learn how to get your scrapers to visit the same URL multiple times with different POST data.

Written by Mihails Murasovs

Submitting forms on .ASPX pages

How to handle pages created with ASP.NET in Web Scraper

Written by Jakub Drobník

Submitting a form with file attachment

How to submit a form with attachment using request-promise.

Written by Jan Čurn

Processing the same page multiple times with different setups in Web Scraper

Solving a common problem with scraper automatically deduplicating the same URLs

Written by Jan Čurn

Scraping data from websites using schema.org Microdata

JavaScript code to automatically extract data using schema.org tags

Written by Jan Čurn