Finding 404 errors with Screaming Frog

What are 404 errors and why should I care about fixing them

A 404 error is an HTTP status code that indicates the server couldn’t find the requested webpage. It’s a standard response code in web browsers, indicating that the client (usually a web browser) was able to communicate with the server, but the server could not find what was requested.

Benefits of fixing 404 errors:

1. Improved User Experience: Users encountering a 404 error can be frustrated or confused. By fixing these errors and providing relevant content or redirection, you enhance user experience and keep visitors engaged on your website.
2. Maintaining SEO Performance: 404 errors can negatively impact your website’s search engine rankings because search engine crawlers interpret them as broken links or missing content. By fixing these errors, you ensure that search engines can properly index your site, leading to better visibility and ranking.
3. Preserving Link Equity: Broken links can disrupt the flow of link equity (or “link juice”) within your website. By redirecting or fixing broken links, you preserve the value of incoming links and distribute it effectively throughout your site.
4. Reduced Bounce Rate: When users encounter a 404 error, they are more likely to leave your site. By minimizing these errors, you reduce your site’s bounce rate, which is a positive signal to search engines and can improve your overall website performance.
5. Enhanced Trust and Credibility: A website that regularly maintains and fixes errors demonstrates professionalism and reliability. Users are more likely to trust and return to websites that provide a seamless browsing experience.

Overall, fixing 404 errors is essential for maintaining a healthy website that provides a positive user experience, preserves SEO performance, and builds trust with visitors and search engines alike.

how to find the 404 errors

This is arguably the hardest part about 404 errors and its the heart of this tutorial.

1. Download and install screaming frog
2. Run the crawl on your site url
3. Filter by status code until 404 descend from the top
4. Select any of the 404’s from the returned results.
5. Look at the footer menu of Screaming Frog for “inlinks”
6. Right click the url from the “from” column
7. Hit open from in browser
8. Search for the anchor text of the broken link by doing a “command f” in your browser window

find 404’s with Screaming Frog Crawl

If you can’t replicate the broken link in Screaming Frog

1. If you cant replicate the link it could be that you are logged in and therefore cant replicate the error. Test the broken link in a non-logged in state.
2. Try search the html source code if the search for the string on the website frontend returns 0 correct results