Screaming Frog SEO Spider

Name: Screaming Frog SEO Spider
Brand: Screaming Frog SEO Spider
Price: 500279 USD

The desktop crawler every technical SEO owns, at $279 a year with no seat games

Screaming Frog SEO Spider is a desktop website crawler for Windows, macOS, and Linux that audits a site for more than 300 technical SEO issues, renders JavaScript with headless Chromium, extracts custom data with XPath, CSS, and regex, pulls in Google Analytics, Search Console, PageSpeed Insights, Ahrefs, Moz, and Majestic data, schedules automated crawls, and can run custom AI prompts against pages while crawling; it is free up to 500 URLs and $279 per licence per year for unlimited crawling.

Visit website

Overview

Screaming Frog is the least fashionable and most universally owned tool in SEO. It is a Java desktop application from a small agency in Henley-on-Thames that has been shipping since 2010, and it has never had a dashboard, a monthly index subscription, or a marketing narrative about AI transformation. What it has is a crawler that will tell you exactly what is on your site, why Google cannot see half of it, and which of your canonical tags point somewhere embarrassing. Almost every technical SEO in the industry has a licence.

The commercial model is the reason it belongs in a small-business buying guide. There is no metered index, no credits, no per-report charge, and no monthly subscription creeping upward. A licence costs $279 per year and unlocks unlimited crawling, constrained only by your machine's memory and disk. The free version crawls 500 URLs with most core features intact, which is enough to audit a small brochure site properly. Multi-licence discounts step down to $265, $249, and $235 each at 5, 10, and 20 licences, and prices are quoted in dollars, pounds, or euros.

Version 24.3 covers considerably more than the broken-link checker most people remember. Headless Chromium rendering handles Angular, React, and Vue sites. Custom extraction with XPath, CSS selectors, and regex pulls prices, SKUs, and structured data out of pages. Crawls can be scheduled and exported automatically to Google Sheets or driven entirely from the command line. Crawl comparison tracks technical progress between audits and maps staging against production. API integrations pull Analytics, Search Console, PageSpeed Insights, Ahrefs, Moz, and Majestic data alongside the crawl, so link and traffic metrics sit next to the technical findings rather than in another tab.

The 2026 addition worth noting is custom AI prompts run against pages during a crawl, supported with OpenAI, Gemini, Anthropic, and locally hosted Ollama models. That last option matters: it means you can run an LLM pass over thousands of pages without sending them to a third party. Separately, Screaming Frog sells a Log File Analyser at $139 per year, free up to 1,000 log lines and one project, which is the companion tool for anyone investigating how search engines actually crawl their site.

Best for

Technical SEOs, developers, agencies, and anyone responsible for a site's technical health who wants a forensic crawler with a flat annual cost and no metering, alongside whatever hosted SEO platform they already pay for.

Not the right fit for

Anyone looking for keyword research, backlink data, or rank tracking; Screaming Frog owns no index and provides none of these, it only reports what it finds by crawling and what other tools' APIs hand it.
Non-technical marketers who want a health score and a prioritised to-do list; the interface is a dense spreadsheet and it assumes you know what a canonical tag does.
Teams that want a hosted tool with client-facing dashboards; this is a desktop application, output is exports and spreadsheets, and there is no client portal or white-label report.
People trying to crawl very large sites on modest hardware; unlimited crawling is limited in practice by your machine's memory and disk, and a million-URL crawl needs real resources and database storage mode configured properly.
Buyers who want AI answer visibility tracking; the AI features here run prompts against your own pages during a crawl, and have nothing to do with whether ChatGPT cites your brand.

How it works

1
You install the desktop application on Windows, macOS on Apple Silicon or Intel, or Linux, enter a starting URL, and it crawls. There is no account setup, no project creation, and no waiting for a hosted crawler queue, because the crawl runs on your own machine using your own bandwidth.
2
Results populate live in a spreadsheet-style interface with tabs for response codes, page titles, meta descriptions, headings, images, canonicals, directives, hreflang, structured data, and more, with a right-hand panel of over 300 detected issue types ranked by category. Filters isolate the problem you care about, and everything exports.
3
For JavaScript sites you switch on headless Chromium rendering and the crawler executes the page as a browser would, discovering content, links, and elements that raw HTML crawling misses entirely. Custom extraction with XPath, CSS selectors, or regex pulls anything else off the page you need, which is how the tool becomes a general-purpose site data extractor rather than only an SEO auditor.
4
Beyond a single manual crawl, you can schedule recurring crawls with automatic export to Google Sheets, drive the whole thing from the command line for CI-style checks, compare two crawls to show what changed between audits, map a staging environment against production, and run custom AI prompts against pages using OpenAI, Gemini, Anthropic, or a local Ollama model while the crawl runs.

Feature breakdown

28 features in 5 modules

Crawling and issue detection

The core job, done more thoroughly than any hosted auditor.

More than 300 detected issues: Broken links and server errors, redirect chains and loops, title and meta description problems, duplicate content detected by MD5 checksum, missing or non-sequential headings, and much else, categorised by severity.
Unlimited crawl on a paid licence: No URL cap and no crawl credits. The only constraint is your machine's memory and storage, which is a completely different cost model from hosted crawlers metering pages per month.
Free crawling up to 500 URLs: The free version retains most core features and is genuinely sufficient to audit a small site properly, which is unusually generous for a paid product.
Directives, canonicals, and hreflang analysis: Full reporting on robots directives, canonical tags including canonicals pointing at error pages, and hreflang implementation across international sites.
Structured data validation: Validates schema markup against Google and Schema.org requirements during the crawl rather than page by page in a separate testing tool.
XML sitemap generation and auditing: Generates sitemaps from a crawl and audits existing ones for orphaned, non-indexable, and missing URLs.

Rendering and extraction

What separates it from a link checker.

Headless Chromium rendering: Executes JavaScript to crawl Angular, React, and Vue sites, discovering content, links, and elements that raw HTML crawling misses entirely, which for a modern single-page application is the difference between an audit and a fiction.
Custom extraction with XPath, CSS, and regex: Pull any HTML off the page: social meta tags, prices, SKUs, author bylines, publication dates, embedded JSON. This turns the crawler into a general-purpose site data extractor.
Custom search: Search the source or rendered text of every crawled page for a string or pattern, which is the fastest way to find a stray tracking tag or a leftover staging URL across a whole site.
Rendered versus raw comparison: See what differs between the HTML served and the DOM after rendering, which is how you diagnose content that Google may never index.
Custom AI prompts during crawl: Run prompts against pages while crawling using OpenAI, Gemini, Anthropic, or a locally hosted Ollama model. The Ollama option means an LLM pass over thousands of pages without sending any of them to a third party.

Automation and comparison

The features that turn a manual audit into a repeatable process.

Scheduled crawls: Run crawls at chosen intervals with automatic export to Google Sheets or a local location, so technical monitoring happens without anyone remembering to do it.
Command line operation: Drive the whole crawler headlessly from the command line, which is how teams wire it into deployment pipelines and CI-style regression checks.
Crawl comparison: Compare two crawls to show exactly what changed between audits, which is the only honest way to demonstrate technical progress over a retainer.
Staging versus production URL mapping: Map a staging environment against live to catch problems before a migration goes out, which is the single most valuable thing this tool does on a replatforming project.
Database storage mode: Switch from memory to disk storage for very large crawls, which is what makes million-URL crawls feasible on a well-specified machine.
Configurable crawl behaviour: Control speed, user agent, robots handling, inclusion and exclusion rules, and authentication, so you can crawl staging environments and rate-limited sites safely.

Data integrations

Brings other tools' data into the crawl rather than replacing it.

Google Search Console: Pulls impressions, clicks, and position data per URL so technical issues sit next to actual performance rather than in isolation.
Google Analytics: Attaches traffic and engagement data per crawled URL, which is how you tell a broken page that matters from one nobody visits.
PageSpeed Insights: Pulls Core Web Vitals and Lighthouse data at scale through the API, rather than testing pages one at a time in a browser.
Ahrefs, Moz, and Majestic APIs: Overlays link metrics from whichever index you already subscribe to, so a page's technical state and its link equity appear in the same row. This is a large part of why Moz's cheap Links API is useful.
Google Sheets export: Scheduled crawls export straight to Sheets, which is how small teams build monitoring without building a dashboard.

Licensing and the Log File Analyser

Flat annual pricing with genuine volume discounts and no metering anywhere.

$279 per licence per year: One flat annual price for unlimited crawling, renewed yearly, with pricing displayed in dollars, pounds, or euros.
Volume discounts: $265 each at 5 to 9 licences, $249 at 10 to 19, and $235 at 20 or more, which is a real agency discount rather than a token one.
No credits, no index subscription: Nothing is metered. You are buying software, not access to a dataset, which is why the total cost of ownership is so different from every hosted tool in this category.
Log File Analyser: A separate desktop tool at $139 per licence per year, free up to 1,000 log lines and one project, for analysing how search engine bots actually crawl your site.
Log File Analyser volume discounts: $125 each at 5 to 9 licences, $115 at 10 to 19, and $99 at 20 or more, on the same annual renewal basis.
Cross-platform desktop: Windows, macOS on both Apple Silicon and Intel, and Linux on Ubuntu and Fedora, with the same feature set on each.

Use cases

4 documented

Technical SEO on a site migration

A replatforming project is due to go live and nobody can say with confidence whether URLs, canonicals, and redirects have been mapped correctly.

Crawl staging with authentication, map staging URLs against production, and produce a definitive list of what breaks before it breaks rather than after.

Developer diagnosing an indexation problem

A React application is losing pages from the index and the hosted SEO tool's crawler reports nothing wrong because it never executed the JavaScript.

Headless Chromium rendering with rendered-versus-raw comparison shows exactly which content and links exist only after execution, which is usually the whole answer.

Agency running monthly technical retainers

Ten client sites need recurring technical audits, and hosted crawlers meter pages per month in ways that make a large ecommerce client unprofitable.

Ten licences at $249 each per year covers unlimited crawling across every client, with scheduled crawls exporting to Google Sheets and crawl comparison demonstrating progress each month.

Content operations lead auditing a large archive

Twenty thousand articles with inconsistent metadata, unknown authorship, and no record of which are thin or duplicated.

Custom extraction pulls bylines and publication dates, duplicate detection flags near-identical content, and custom AI prompts running against a local Ollama model classify the archive without sending any of it outside the network.

Pricing

from Free up to 500 URLs, then $279 per licence per year

Flat annual per-licence software fee with unlimited crawling, no credits and no metered index, plus a separately licensed Log File Analyser and progressive volume discounts.

Plan	Price	Includes
Free	$0 no time limit	500 URL crawl limit Broken link and redirect detection Page title and metadata analysis XML sitemap generation Google Analytics and Search Console integration Genuinely sufficient to audit a small brochure site. Most of what is missing is scale, scheduling, custom extraction, and JavaScript rendering.
SEO Spider licence	$279 per licence per year	Unlimited crawling, bounded only by your machine Headless Chromium JavaScript rendering Custom extraction with XPath, CSS, and regex Scheduled crawls and command line operation Crawl comparison, structured data validation, and full API integrations One flat price, renewed annually, with no metering of any kind anywhere in the product.
SEO Spider volume licensing	$265 to $235 per licence per year	$265 each at 5 to 9 licences $249 each at 10 to 19 licences $235 each at 20 or more licences Identical feature set across all licences A real agency discount, and cheap enough that whole teams get licensed rather than sharing one machine.
Log File Analyser	$139 per licence per year	Unlimited log events and projects Free tier limited to 1,000 lines and 1 project Bot verification and error detection $125, $115, and $99 per licence at 5, 10, and 20 licences A separate purchase, and the right companion when you need to know how search engines actually crawl rather than how you think they do.

Billing notes

This is a software licence, not a data subscription. Nothing is metered: no crawl credits, no report credits, no index access fee.
Licences last one year and require renewal. Each user needs their own licence, but at $279 that is a fraction of a per-seat charge on any hosted suite.
Volume discounts step down progressively to $265, $249, and $235 per licence at 5, 10, and 20 licences, so agencies licensing a whole team pay meaningfully less per person.
The free version has no time limit and is capped at 500 URLs, so a small site can be audited properly at zero cost indefinitely.
Prices are displayed in US dollars, pounds sterling, and euros, so buyers outside the UK are not forced into currency conversion.
The Log File Analyser is a separate product at $139 per licence per year with its own volume discounts, free up to 1,000 log lines and one project.
Unlimited crawling is limited in practice by your own hardware. Very large crawls require database storage mode and a machine with adequate memory and disk, which is a real cost that does not appear on the invoice.

Value assessment: Screaming Frog is the best value purchase in this entire category, and the comparison is not close because it is not the same kind of purchase. $279 per year buys unlimited crawling forever, against hosted auditors metering 250,000 or 2 million pages per month inside subscriptions costing four to twenty times as much. A ten-person agency licences everyone for $2,490 a year, roughly what one Ahrefs Advanced seat plus its per-user surcharge costs over the same period. The reason the price can be this low is that Screaming Frog owns no index and rents you no data; it sells you a program that runs on your machine. That is also its limitation. You still need a keyword and backlink subscription somewhere, and the correct way to think about this is that the crawler is the cheap part of your stack that should never have been a subscription in the first place.

Strengths & limitations

Strengths

Flat $279 per year with unlimited crawling and no metering anywhere, against hosted crawlers that ration pages per month inside far more expensive subscriptions.
More than 300 detected issue types with genuinely forensic depth, including canonical, hreflang, directive, and structured data analysis that hosted auditors summarise rather than expose.
Headless Chromium rendering makes it one of the few tools that will audit a modern JavaScript application honestly.
Custom extraction with XPath, CSS selectors, and regex turns it into a general-purpose site data extractor, which is why non-SEO teams end up using it too.
Scheduling, command line operation, and crawl comparison make repeatable technical monitoring possible without building anything.
API integrations with Search Console, Analytics, PageSpeed Insights, Ahrefs, Moz, and Majestic put technical findings and performance data in the same row.
Custom AI prompts can run against a locally hosted Ollama model, so an LLM pass over thousands of pages never leaves your network.
A free 500-URL version with no time limit, and volume licence discounts down to $235 each, so both individuals and agencies are treated fairly.

Limitations

It owns no index. No keyword data, no backlink data, no rank tracking, and no competitive research, so it is always a second tool rather than a first one.
The interface is a dense spreadsheet with no health score and no prioritised action list, which makes it hostile to non-technical users.
Unlimited crawling is bounded by your own hardware, and large crawls require configuring database storage mode and having the memory and disk to back it.
No hosted dashboards, no client portal, and no white-label reporting; output is exports and spreadsheets.
The AI features run prompts against your own pages during a crawl and have nothing to do with AI answer visibility or citation tracking, despite the surface similarity in language.
Being a desktop application, crawls run on your machine and your bandwidth, and a long crawl occupies the computer you were planning to work on.
Licences require annual renewal per user, so a team of ten is a $2,490 recurring line rather than a one-time purchase.

Head-to-head comparisons

4 alternatives

Screaming Frog SEO Spider vs Moz Pro

from $49 per month (Starter), or $39 per month billed annually

Different halves of the same job. Moz Pro brings the link index, keyword data, authority metrics, and ongoing monitoring that Screaming Frog cannot; Screaming Frog crawls far more forensically than Moz Site Crawl ever will, for a flat $279 a year with no page limits. Screaming Frog even pulls Moz link metrics into the crawl through the API. Most technical teams run both, and the combined cost is still below one Ahrefs subscription.

Full Screaming Frog SEO Spider vs Moz Pro comparison

Screaming Frog SEO Spider vs SISTRIX

from 119 EUR per month (START)

SISTRIX gives you thirteen years of visibility history, keyword data, and AI answer tracking in a hosted platform from 119 euros per month; Screaming Frog gives you a crawler that goes deeper than any hosted auditor for $279 a year. SISTRIX tells you what happened to your visibility, Screaming Frog tells you why the site is built the way it is. European technical teams commonly hold both.

Full Screaming Frog SEO Spider vs SISTRIX comparison

Screaming Frog SEO Spider vs SEOTesting

from $50 per month (Single Site)

Both are cheap tools that make you more rigorous rather than more informed. Screaming Frog establishes the technical state of a site; SEOTesting establishes whether a change you made actually caused a result, using controlled tests on Search Console data. Neither replaces the other, and together they cost less per year than two months of a mid-tier hosted suite.

Full Screaming Frog SEO Spider vs SEOTesting comparison

Screaming Frog SEO Spider vs Ranktracker

from $39 per month (Launch, billed annually at $312 per year)

Screaming Frog audits, Rank Tracker monitors positions and produces client-facing reports. They do not overlap at all. A small agency typically needs a crawler and a tracker as separate purchases, and buying both is cheaper than a single suite that does each of them less well.

Full Screaming Frog SEO Spider vs Ranktracker comparison

Implementation & onboarding

Setup time: Ten minutes to install and run a first crawl. Configuring it properly for a large or JavaScript-heavy site, including rendering, storage mode, exclusions, and API connections, takes a couple of hours the first time and is then reusable as a saved configuration.
Learning curve: Steep for a non-specialist and shallow for anyone who already understands technical SEO. The interface presents raw data rather than recommendations, which is exactly what an expert wants and exactly what a marketer without a technical background will find impenetrable.
Onboarding: Fully self-serve. Download, run the free version up to 500 URLs, and buy a licence when you hit the cap. Screaming Frog's own user guide and tutorials are thorough and free, and the tool is so widely used that third-party training material is abundant.
Migration notes: Nothing to migrate, since crawls are local files. Saved configurations and custom extraction rules transfer between machines as exported config files, which is how agencies standardise audits across a team. If you are replacing a hosted auditor, note that you lose its historical issue tracking, so run a baseline crawl and keep it for comparison.

Platform, API & security

Platforms: WindowsmacOS (Apple Silicon and Intel)Linux (Ubuntu and Fedora)Command line interface
API: No hosted API of its own, since the product is desktop software. It consumes third-party APIs including Google Search Console, Google Analytics, PageSpeed Insights, Ahrefs, Moz, and Majestic, and can be driven headlessly from the command line for integration into pipelines.
Compliance: GDPR (UK company)
Data residency: Crawls run on your own machine and crawl data stays local unless you export it, which is a substantive privacy advantage over hosted crawlers. Custom AI prompts can be pointed at a local Ollama model so page content never leaves your network.
SSO: Not applicable to a desktop licence.
Security notes: Because the software runs locally, crawl data and any extracted page content remain under your control rather than being processed on a vendor's infrastructure. This makes Screaming Frog viable in environments where sending site content to a hosted SEO platform would not be permitted, particularly with local model support for the AI features.

Support & resources

Channels: Email support for licence holdersSupport ticketing through the website
Documentation: A thorough user guide, an FAQ, tutorials, and detailed release notes for each version, all free and public.
Community: Very large practitioner community with extensive third-party tutorials, configuration recipes, and shared custom extraction snippets; the tool is effectively an industry standard.

Company

Founded: 2010
Headquarters: Henley-on-Thames, Oxfordshire, United Kingdom
Ownership: Independent, founder-led and privately held
Founders: Dan Sharp
Employees: Small team, reported in the low tens
Funding: No outside funding. Screaming Frog is an unfunded UK company operating as both a search marketing agency and a software vendor, self-funded since 2010.

Timeline

2010Founded in Henley-on-Thames by Dan Sharp as a search marketing agency, with the SEO Spider built initially as an internal tool.
2011The SEO Spider is released publicly with a free 500 URL tier and a paid unlimited licence, a structure that has never fundamentally changed.
2016Adds headless rendering so JavaScript-driven sites can be crawled as a browser sees them rather than as raw HTML.
2018Launches the Log File Analyser as a separate product for examining how search engine bots actually crawl a site.
2021Adds scheduled crawling with automatic export, crawl comparison, and command line operation, turning manual audits into repeatable monitoring.
2026Version 24.3 ships with custom AI prompts run against pages during a crawl, supported by OpenAI, Gemini, Anthropic, and locally hosted Ollama models.

Integrations

Google Search Console
Google Analytics
Google PageSpeed Insights
Google Sheets (scheduled export)
Ahrefs API
Moz API
Majestic API
OpenAI, Google Gemini, Anthropic, and Ollama for custom AI prompts
Command line for pipeline integration

Frequently asked questions

10 questions

What is Screaming Frog SEO Spider?

It is a desktop website crawler for Windows, macOS, and Linux that audits a site for more than 300 technical SEO issues. It renders JavaScript with headless Chromium, extracts custom data with XPath, CSS, and regex, pulls in Search Console, Analytics, PageSpeed, Ahrefs, Moz, and Majestic data, schedules crawls, and compares audits over time.

How much does Screaming Frog cost?

The free version crawls up to 500 URLs with no time limit. A paid licence is $279 per year for unlimited crawling, with volume discounts to $265 each at 5 to 9 licences, $249 at 10 to 19, and $235 at 20 or more. The separate Log File Analyser is $139 per year, free up to 1,000 log lines and one project.

Is the free version of Screaming Frog actually useful?

Yes, more than most free tiers in this category. It keeps most core features including broken link detection, metadata analysis, XML sitemap generation, and Google Analytics and Search Console integration, and 500 URLs is enough to audit a small brochure site completely. What you lose is scale, scheduling, custom extraction, and JavaScript rendering.

Does Screaming Frog do keyword research or backlink analysis?

No. It owns no index at all. It reports what it finds by crawling your site, plus whatever data you connect through third-party APIs such as Ahrefs, Moz, or Majestic. This is why it is always a companion purchase alongside a hosted suite rather than a replacement for one, and why the price can be as low as it is.

Can Screaming Frog crawl JavaScript sites?

Yes, using headless Chromium rendering, which executes the page as a browser would and discovers content, links, and elements that raw HTML crawling misses. On Angular, React, or Vue applications this is the difference between a real audit and a misleading one, and it is a paid-licence feature.

How large a site can it crawl?

Officially unlimited on a paid licence, but practically bounded by your own hardware. For very large crawls you switch from memory to database storage mode and need adequate disk and RAM. Million-URL crawls are entirely feasible on a well-specified machine, which is a genuine cost that never appears on the invoice.

What are the AI features in Screaming Frog?

You can run custom AI prompts against pages while crawling, using OpenAI, Gemini, Anthropic, or a locally hosted Ollama model. That is useful for classifying or summarising large archives at scale. It is not AI answer visibility tracking, and it will not tell you whether ChatGPT cites your brand, which is a different product category entirely.

Does each person need their own licence?

Yes, licences are per user and renew annually. At $279 each, falling to $235 at twenty or more, that is still far cheaper than the per-seat surcharges on hosted suites, where an additional user can cost $40 to $80 per month. Agencies routinely licence entire teams rather than sharing one installation.

What is the Log File Analyser and do I need it?

It is a separate desktop tool at $139 per licence per year, free up to 1,000 log lines and one project, that analyses server logs to show how search engine bots actually crawl your site: which URLs they request, how often, and what responses they receive. Most small sites do not need it. Anyone diagnosing crawl budget problems on a large site does.

Is Screaming Frog private, given it crawls my site?

More private than a hosted alternative. The software runs on your own machine and crawl data stays local unless you export it, and the AI prompt feature can be pointed at a locally hosted Ollama model so page content never leaves your network. That makes it viable in environments where uploading site content to a hosted SEO platform would not be permitted.

Editorial verdict

Screaming Frog is the tool in this category whose pricing makes the most sense, because it is honest about what it is selling. There is no index to rent, so there is nothing to meter, so a licence costs $279 a year and crawls as much as your laptop can handle. Set that against hosted auditors rationing 250,000 pages a month inside subscriptions costing several times more and the arithmetic barely needs stating. What you get is genuinely forensic: headless Chromium rendering that audits JavaScript applications honestly, custom extraction with XPath and regex, scheduled crawls and command line operation, crawl comparison for demonstrating progress, and in 2026 the option of running AI prompts against your pages using a local model so nothing leaves your network. What you do not get is any data of its own, any dashboard, or any help for a non-technical user. Buy it as the crawler in a stack that already has a keyword and link subscription somewhere else, start with the free 500-URL version, and licence the whole team when you outgrow it.

Written by the SaaSTracker editorial team. Awards, when shown, are judged against the published criteria in our methodology.