This article contains a summary of the essentials for tracking down and fixing failing Actors.
You can see a detailed breakdown and code examples in our documentation.
Possible problems
The most common (but not all) causes for errors in Actor runs include:
Changes in website layout or data feed.
Website changes layout depending on location or uses A/B testing.
The website begins blocking you (recognizes you as a bot).
The site uses dynamic data loading, so the code only works occasionally.
Mistakes in your code.
The Actor works locally but not on the Apify platform.
You lose access to Apify proxy (your trial expires).
The errors appear after a dependency update (harder to debug).
How to analyze the error
1. Logging
Use structured logs that focus on numbers, e.g. how many items you extracted from a page. Add each page's URL to each log so you can open it quickly and check.
If your code crashes, however, your usual logs will not run. In this case, use try/catch blocks in your code to find out where the error is.
2. Saving snapshots
Snapshots can be either screenshots or a copy of the page's HTML. They can help you determine if a page's layout has changed, if you have been blocked, or if the page is empty. Check our docs for examples of how you can save snapshots.
3. Error reporting
For large projects, snapshots and logging may not be enough. In this case, consider creating a named dataset in which your error reports accumulate. See an example.