That's a lot of features and we know how overwhelming it can feel for our new users. You, as a user, care much more about extracting high-quality data that can power your business than our platform's technical details. This article provides a short summary of each of our tools, so to help you choose the right one.
Apify Store - ready-made solutions for popular sites. The quickest way to get your data or automate your process.
Apify freelancers - get a custom solution from a developer in the Apify community. For more specific use cases.
Enterprise - premium solution at any scale from a dedicated data expert here at Apify.
Apify Store is the quickest path to the data you need. It's full of open-source actors (Google Search, Booking, Instagram, etc.) that you can use right away without any upfront commitment. Just sign up for a free account, set your input, press Run, and wait for the data to flow in.
If your use case is more specific, you may need a more tailored solution. In this case, you can either develop your tools yourself (or with your team) or let our experts (or partners) prepare a custom solution.
Apify freelancers are members of the Apify community who can provide custom web scraping and automation solutions at a fair price. You can think of it as a more specialized Freelancer.com. The developers know our platform well and have plenty of expertise using it.
All you need to do is provide a specification. Developers will bid their price and you can choose the offer that fits you best. Just head over to the Apify Discord and post your project.
Apify for Enterprise
If you want a more organized approach, we also offer a premium service called Apify for Enterprise. As our enterprise customer, we will assign you a dedicated data expert who will manage the whole project, send you periodic reports, and communicate with you via whatever tools you prefer (email, Slack, UberConference, etc.). We will sign a contract describing a specific SLA, integrate into your platform, and maintain the project long-term (if needed).
If you aren't sure whether an Apify freelancer or an Enterprise solution is best for you, you can compare them here.
Develop your own solution
Fortunately, Apify provides you with some of the best development tools and documentation on the market. In the rest of this article, we will discuss these tools and their particular pros and cons.
Scrapers vs. SDK
Since scraping and automation come in various forms, we decided to build not just one, but three scrapers. This way, you can always pick the right tool for the job.
Let's discuss each particular tool and its advantages and disadvantages.
Web Scraper (apify/web-scraper) crawls a website's pages using a headless Chrome browser and executes some code (we call it the
pageFunction) on each page.
The UI of Cheerio Scraper (apify/cheerio-scraper) is almost the same as that of Web Scraper. What changes is the underlying technology. Instead of using a browser to crawl the website, it fires a series of simple HTTP requests to get the page's HTML. Your code then executes on top of that HTML with help from the Cheerio parser, which is essentially JQuery for servers. The main goal here is speed (and therefore cost-effectiveness).
Unlike the two previous scrapers, Puppeteer Scraper (apify/puppeteer-scraper) doesn't focus primarily on simplicity, but provides a wider variety of powerful features. You have the whole Puppeteer library for managing headless Chrome at your disposal.
+ powerful Puppeteer functions (methods on the
- more complex
+ complete freedom, maximum flexibility
- requires that you write some boilerplate code
- more complex, higher chance of making mistakes (the flip side of freedom)
Whatever tool you choose, don't feel trapped by your initial choice. Play and experiment: you can easily transfer large parts of the code between all of our tools. Over time you'll get a feeling for when to use what and you may also find your personal favorite.
If you feel like our scrapers aren't doing what they should do, you can always report an issue on their public GitHub page.
Just remember you shouldn't post cases when your own code fails for any reason. If that is the case and you cannot progress despite your best efforts, it's best to post the problem on StackOverflow with an
apify tag. Our experts go through these posts periodically and will leave you a reply anybody can learn from. Or you can always just contact our support team!