All Collections
Developing Actors
How to analyze and fix errors in your actors
How to analyze and fix errors in your actors

Your scrapers randomly crash? They worked for a few months and then stopped? Understand how to analyze and fix problems.

Kirill Turin avatar
Written by Kirill Turin
Updated over a week ago

This article contains a summary of the essentials for tracking down and fixing failing actors.

You can see a detailed breakdown and code examples in our documentation.

Possible problems

The most common (but not all) causes for errors in actor runs include:

  • Changes in website layout or data feed.

  • Website changes layout depending on location or uses A/B testing.

  • The website begins blocking you (recognizes you as a bot).

  • The site uses dynamic data loading, so the code only works occasionally.

  • Mistakes in your code.

  • The actor works locally but not on the Apify platform.

  • You lose access to Apify proxy (your trial expires).

  • The errors appear after a dependency update (harder to debug).

How to analyze the error

1. Logging

Use structured logs that focus on numbers, e.g. how many items you extracted from a page. Add each page's URL to each log so you can open it quickly and check.

If your code crashes, however, your usual logs will not run. In this case, use try/catch blocks in your code to find out where the error is.

2. Saving snapshots

Snapshots can be either screenshots or a copy of the page's HTML. They can help you determine if a page's layout has changed, if you have been blocked, or if the page is empty. Check our docs for examples of how you can save snapshots.

3. Error reporting

For large projects, snapshots and logging may not be enough. In this case, consider creating a named dataset in which your error reports accumulate. See an example.

Did this answer your question?