Help us make the web more programmable ✨Join our team
Extracting data
Use Apify actors or scrapers to extract any data you need from any website.
+5
14 articles in this collection
Written by Jan Čurn, Jakub Drobník, Marek Trunkát and 5 others
Written by Jan Čurn, Jakub Drobník, Marek Trunkát and 5 others
Apify's free Google SERP API
How to stay up to date on search results with a Google SERP API
Written by Sapiens Malti
Waiting for dynamic content
You load the page. You execute the correct selectors. Everything should work. It doesn't? Learn how to wait for dynamic loading.
Written by Lukáš Křivka
Request labels and how to pass data to other requests
How to handle request labels in Apify actors with Cheerio or Puppeteer Crawler
Written by Václav Růt
Scraping a list of URLs from a Google Sheets document
Learn how to crawl a list of URLs specified in a Google Sheets document using one of the Apify web scraping actors.
Written by Jan Čurn
How to scrape pages with shadow DOM
Shadow DOM enables the isolation of web components, but causes problems for web scrapers. Here's an easy workaround.
Written by Marek Trunkát
Scraping iframes with Puppeteer
How to get information from inside iframes using Puppeteer
Written by Lukáš Křivka
When to use Puppeteer Scraper
Choosing between Web Scraper and Puppeteer Scraper can be difficult. We explain the important differences to help you pick the right tool.
Written by Ondra Urban
How to scrape a website using sitemap.xml and Apify SDK
Website with proper sitemap.xml = free jackpot for every web scraper
Written by Marek Trunkát
Submit form with file attachment using Puppeteer
How to submit a form with attachment using Puppeteer.
Written by Jan Čurn
Send multiple POST requests to the same URL
Learn how to get your scrapers to visit the same URL multiple times with different POST data.
Written by Mihails Murasovs
Submitting forms on .ASPX pages
How to handle pages created with ASP.NET in Web Scraper
Written by Jakub Drobník
Submitting a form with file attachment
How to submit a form with attachment using request-promise.
Written by Jan Čurn
Processing the same page multiple times with different setups in Web Scraper
Solving a common problem with scraper automatically deduplicating the same URLs
Written by Jan Čurn
Scraping data from websites using schema.org Microdata
JavaScript code to automatically extract data using schema.org tags
Written by Jan Čurn