Screaming Frog is one of the most important tools in an SEO’s toolkit.

It crawls any domain or URL(s) placed into it, returning raw data from the pages source codes. 

Users can also use their API function to connect with tools such as Google Analytics, Google Search Console, Ahrefs, Majestic, Moz & Page Speed Insights in order to obtain metrics for each individual URL returned in the crawl results.

Crawls can be done by entering a URL in & letting the bot parse through all of the links it finds on the site from the entry-point, by uploading a list of specific URLs on a domain to crawl & in a manner called SERP mode, where you can count pixels that will be in your meta data to see how they might render in search.

Screaming Frog also enables you to search for regular expressions in your page content, source code & more, helping you to debug your site.

XML sitemaps can also be generated for HTML pages, and image files by using the XML Sitemap Export option from a standard crawl.

Crawls are able to be scheduled & automated, but in those instances API data may not be readily accessible, as they typically require you to connect to your desired accounts & select the fields of information that you are looking for before starting the crawl.

Internal & External files that are used to render your website will be found & returned in each crawl, which can aid in resource debugging & optimizing for page load speed.

They also feature a number of valuable Bulk Exports & Reports, where you can find information on the status of items related to Structured Data Markups, HREF-LANG, Directives, Re-Direct Chains & more.

For smaller websites, using the free version may be ok, but for sites that will be using more than 500 returned resources it is beneficial to purchase a subscription, and well worth it, as it is the most comprehensive tool that is a must have for your tool set.