Discover the hidden gems of this powerful audit tool.
If you want to grow online, SEO is essential. It helps you become visible to the right audience and stay ahead of competitors. Many tools only provide a surface-level view. Screaming Frog goes deep. It crawls your entire website and uncovers what’s holding back your growth: broken links, duplicate content, missing metadata, messy structures.
The real power lies in features many users overlook. In this blog, we reveal the secret features that make audits more complete, faster, and more valuable.
For many SEO specialists, Screaming Frog is the starting point of a technical audit. The tool crawls your entire site and shows how pages, links, and metadata interconnect. This allows you to see at a glance where improvements are needed.
By default, you get insights into:
Extra benefit: you can easily connect Screaming Frog to Google Analytics, Search Console, and PageSpeed Insights. This lets you combine crawl results with performance data and context.
The real gold, however, is in the hidden features that often go unused.
1. COMPARE MULTIPLE XML SITEMAPS
Upload an XML sitemap (Mode → List → Upload → Download XML Sitemap) and compare multiple sitemaps via ChatGPT.
See which URLs are unique and which are missing. This keeps you organized and prevents important pages from being missing or duplicated.
2. COMPARE CRAWLS
Compare two crawls (File → Crawls → Compare), for example after a redesign or migration.
See which pages are new, deleted, or changed. Understand the impact of changes immediately and quickly adjust where needed.
3. USER-AGENT SWITCHING
Simulate how search engines see your site (Configuration → User-Agent).
Discover if important pages are accessible and if dynamic content is displayed correctly.
4. CUSTOM EXTRACTION
Extract specific data from the source code (Configuration → Custom → Extraction).Automate collecting prices, reviews, or structured data. Quickly gain insights into content that would otherwise be hard to analyze and save manual work.
5. VISUALIZE YOUR SITE ARCHITECTURE
Create a visual overview (Visualisations → Crawl Tree Graph / Force-Directed Diagram).
See how pages are organized, where content is buried deep, and where bottlenecks occur. Improve internal navigation and structure, making your site more user- and SEO-friendly.
6. DISCOVER ORPHAN PAGES
Find pages that get traffic but aren’t linked internally (Configuration → API Access → Google Analytics / Search Console).
These pages lack internal support. By making them visible, you increase the reach and value of content that would otherwise remain hidden.
7. JAVASCRIPT RENDERING
Let Screaming Frog crawl as Google sees dynamic content (Configuration → Spider → Rendering → JavaScript).
Discover which content only becomes visible after rendering. This prevents important dynamic elements from being unindexed.
8. SPELLING AND GRAMMAR CHECK
Check titles, meta descriptions, and headings (Configuration → Spelling & Grammar).
Keep metadata error-free and professional, making your site look trustworthy to both visitors and search engines.
9. ANALYZE INLINKS AND OUTLINKS
See which pages get the most internal links and which anchor texts are used (click on a URL → Tabs: Inlinks and Outlinks). Gain insight into how link value flows through your site and optimize the internal link structure to maximize authority.
10. TEST CUSTOM ROBOTS.TXT
Test crawler rules safely (Configuration → Robots.txt → Settings → Custom).
Experiment without risk and ensure that important pages remain accessible.
Since version 21.0, you can connect Screaming Frog directly to OpenAI, Gemini, Anthropic, and Ollama (Configuration → API Access → AI).
Automatic prompts run during crawls. Choose content to analyze (HTML, body text, custom extractions) and output such as alt text, intent detection, sentiment, or summaries. Crawling + AI = scalable, consistent SEO.
Compare similar content (Configuration → Content → Embeddings)
Automatic image descriptions
AI creates compelling descriptions instantly
Screaming Frog SEO Spider is more than just a standard audit tool. It’s an all-rounder that helps you optimize your website down to the smallest details. The possibilities are virtually endless. By discovering and using these hidden features intelligently, you gain insight into what’s holding your site back. By implementing targeted improvements, you can significantly boost your visibility and performance in search engines. For anyone serious about SEO, Screaming Frog is a secret weapon that gives your digital growth a substantial boost.
Our Head of SEO will show you how Screaming Frog can accelerate your content and performance. Schedule your demo or strategy session today.
Doing SEO yourself is possible, but it requires a lot of knowledge, time, and the right tools. By outsourcing SEO to specialists, you benefit from expertise, consistency, and faster, more sustainable results.
Yes, ChatGPT can create SEO texts, but without the right strategy and optimization, you won’t get the most out of it. The best results come from combining AI with human creativity and SEO expertise.
he four types of SEO are technical SEO, on-page SEO, off-page SEO, and local SEO. Together, they form a complete approach to improving your visibility and authority.
Absolutely! From our office in Rotterdam, we help businesses in the region and beyond improve their visibility on Google. We combine local market knowledge with broad SEO expertise to deliver the best possible results.
