Netpeak Spider
Netpeak Spider is an SEO crawler for a day-to-day SEO audit, fast issue check, comprehensive analysis, and website scraping.
With Netpeak Spider you can analyze incoming and outgoing links, find broken links and redirects, consider indexation instructions, and escape duplicate content: Titles, Meta Descriptions, H1 Headers, full pages, etc. A tool can calculate internal PageRank to improve website linking structure, and you can set custom rules to crawl either the entire website or its certain part.
Learn more
Screaming Frog SEO Spider
The Screaming Frog's SEO Spider is a website spider that can help you improve your onsite SEO by extracting data and auditing for common issues. You can download and crawl 500 URLs free of charge, or you can purchase a license to unlock advanced features and remove the limit. The SEO Spider is a flexible and powerful site crawler that can crawl small and large websites quickly and allows you to analyze the results in real time. It collects key information on the site to help SEOs make informed decisions. Instantly crawl a website and identify broken links (404s), and server errors. Bulk exports the errors and source URLs to fix or send to a developer. Locate temporary and permanent redirects, identify loops and redirect chains, and upload a list to review during a site migration. Analyze page titles, meta descriptions, and other information during a crawl to identify duplicates or pages that are too long, too short, missing, not present, or duplicated throughout your site.
Learn more
TechSEO360
TechSEO360 is a complete technical SEO crawler software tool that can:
- Fix broken redirects, broken links and broken canonical refers
- Find pages that have thin content, duplicate titles, duplicate headings, duplicate meta, and similar content.
Analyze keywords across pages or entire websites.
- Create sitemaps in HTML, XML, image, and video, including hreflang information.
Integrate with 3rd party data exports such as Apache logs, Google Search Console, and many more. TechSEO360 can then combine the data from these sources to create custom reports that can be exported to Excel or CSV.
- Explore large websites.
- Search Javascript code for linking
AJAX mode is recommended for websites that have this requirement.
- For analysis and output, configure the crawler separately with limit-to and exclusion filters.
- Use the command line interface to automate and schedule most of your work.
Learn more
Webbee SEO Spider
Webbee, a desktop-based SEO spider, crawls your website in accordance with major search engine bots. It scans every corner of your website to find potential opportunities and issues that could be addressed. Get it now to learn the exact steps you need to make your website a traffic magnet. Webbee SEO Spider crawls your site in accordance with major search engines' guidelines. It collects all information from your website and creates a search engine strategy for you website. Our spider can crawl titles, headings (h1-h6 with their frequency), http URLs, status codes (20 OK, Redirects), 404 pages, server error pages), page types (images html, CSS, JS flash, PDF), GA codes and meta robots. It can also crawl all internal links, all links externally, links frequency to internal pages, and all anchor texts and their frequency.
Learn more