Extracting data
Use Apify actors or scrapers to extract any data you need from any website.
+2
10 articles in this collection
Written by Jan Čurn, Petr Čermák, Marek Trunkát and 2 others
Written by Jan Čurn, Petr Čermák, Marek Trunkát and 2 others
Waiting for dynamic content
You load the page. You execute the correct selectors. Everything should work. It doesn't? Learn how to wait for dynamic loading.
Written by Lukáš Křivka
Request labels and how to pass data to request in Puppeteer
How to handle request labels in Apify actors with Puppeteer
Written by Václav Růt
Scraping a list of URLs from a Google Sheets document
Learn how to crawl a list of URLs specified in a Google Sheets document using one of the Apify web scraping actors.
Written by Jan Čurn
How to scrape pages with shadow DOM
Shadow DOM enables the isolation of web components, but causes problems for web scrapers. Here's an easy workaround.
Written by Marek Trunkát
Scraping iframes with Puppeteer
How to get information from inside iframes using Puppeteer
Written by Lukáš Křivka
Scraping website using sitemap.xml and Apify SDK
Website with proper sitemap.xml = free jackpot for every web scraper
Written by Marek Trunkát
Submit form with file attachment using Puppeteer
How to submit a form with attachment using Puppeteer.
Written by Petr Čermák
Crawl multiple pages with the same URL and different POST data
Crawlers skip pages that have the same URL but only differ in POST data. Learn how to make the crawler visit all such pages.
Written by Jan Čurn
Submitting a form with file attachment
How to submit a form with attachment using request-promise.
Written by Petr Čermák
Scraping data from websites using schema.org Microdata
JavaScript code to automatically extract data using schema.org tags
Written by Jan Čurn