What Is SiteGraph? Inside the Free Engine Mapping the Internet, One Website at a Time

A few months ago I was staring at a competitor's website, doing the thing every founder secretly does: right-click, "View Page Source," and squint. I wanted to know what they were running. Was it Next.js or plain React? Who was hosting them? Were they even bothering with basic security headers? I could piece some of it together by hand, but the good tools that answer this properly all wanted a credit card before they'd tell me more than three things.

That annoyance is the entire origin story of SiteGraph. Not a grand vision statement — just a very ordinary frustration that a huge amount of useful information about any website is technically public, technically free to fetch, and yet practically locked behind paywalls because nobody had bothered to build the free version properly. So we built it.

What Is SiteGraph, in One Sentence

SiteGraph is a free, real-time website intelligence platform: you give it a domain, and in seconds it hands back a full technical, SEO, security, and performance profile of that site — and then keeps that profile as a permanent, public page anyone can browse later.

No sign-up wall to see the first result. No "unlock full report" button. The report you get on scan one is the same report a paying BuiltWith customer would expect, because we don't buy the data from anyone — we go and look ourselves, the same way a browser does.

Why "Graph," Not Just "Scanner"

The name is the whole idea. A scanner gives you one report about one site and then forgets it ever happened. A graph remembers, and connects. Every site SiteGraph analyzes becomes a node, and every technology, hosting provider, CDN, and country it's linked to becomes an edge you can actually walk along.

Concretely, that means from any scanned site's report page you can jump to:

• Every other site using the same CMS or framework — see who else is on WordPress, Next.js, or Shopify
• Every site on the same hosting provider or CDN — useful for spotting infrastructure patterns in an industry
• Every site in the same country — a regional slice of the web
• A side-by-side comparison of any two domains' tech stack, SEO, and security posture

That's the graph. It's not a metaphor bolted on for the name — it's the actual data model, and it's what turns a pile of individual scans into something you can explore rather than just look up.

What Actually Happens When You Scan a Site

Behind the two-second wait, a real crawl runs. Nothing here is guessed or bought from a third-party data broker — every field on the report comes from data SiteGraph fetched itself, moments (or, for a cached recent scan, hours) before you looked:

• DNS resolution — A/AAAA, MX, TXT (including SPF, DKIM, DMARC), NS, and CAA records
• TLS/SSL inspection — certificate issuer, validity window, days to expiry, TLS version, SAN domains
• Raw HTTP requests — response headers, security headers (HSTS, CSP, X-Frame-Options, and more), cookies and their Secure/HttpOnly/SameSite attributes
• HTML parsing — meta tags, Open Graph and Twitter Card tags, canonical URLs, heading structure, JSON-LD structured data
• Technology fingerprinting — script signatures, CSS class conventions, generator tags, and DOM patterns matched against a growing signature library
• robots.txt and sitemap.xml — whether they exist, whether they're wired together correctly, and whether any rule looks like it's accidentally blocking important content
• Hosting and ASN lookup — who actually owns the IP the site resolves to, via free WHOIS-over-DNS data

All of that gets rolled into an overall score and a full JSON report, and the page it produces is designed to be genuinely useful on its own — not a teaser for something you have to pay to unlock.

The Architecture, in Plain English

Under the hood, SiteGraph is a small, deliberately boring stack: a Go API that takes your request, a Redis queue that holds the job, a fleet of Go workers that actually go out and crawl the target site, and a Postgres database that remembers every result forever. A Next.js frontend sits in front of all of it.

Think of it like a mail-sorting facility. The API is the front counter — it takes your request and hands you a ticket. Redis is the conveyor belt. The workers are the people at the sorting stations who actually open the envelope (the target website) and read what's inside. Postgres is the archive room where every opened envelope gets filed, indexed, and made searchable forever. Because each piece is a separate, independently deployable process, we can add more sorting stations (workers) the moment the conveyor belt starts backing up, without touching the front counter at all.

It's one Go module with several entry points rather than a sprawl of microservices — simple enough that the whole system is understandable by one person, which matters a lot more than it sounds like it should.

Real Numbers From a Real, Running System

As of writing this, SiteGraph has profiled 5,987 distinct websites across 11,070 total scans — over 6,000 of those scans in the last 24 hours alone, most of them re-checks the scheduler runs automatically to keep old reports honest. The average overall score across every site in the index currently sits at 83.5 out of 100.

The technology numbers threw me a little. The single most-detected JavaScript library across every scanned site right now isn't React, Vue, or Svelte — it's jQuery, still running on well over a thousand sites, a library plenty of people declared dead a decade ago. Google Analytics remains the single most common script on the web by a wide margin. Nginx edges out Apache as the most-detected web server. And WordPress is still, comfortably, the most-detected CMS on the internet — nearly five times more common in our index than the next runner-up, Drupal.

Meanwhile Next.js already shows up on more scanned sites than bare React does — a small but telling sign of how much of the "React ecosystem" has quietly moved into meta-frameworks rather than staying as raw library usage. None of this is a press release; it's just what a live index of nearly 6,000 real websites currently looks like, updating every day as the scheduler re-checks old sites and new ones get submitted.

Every Scan Becomes a Public, Permanent Page

This is the part I think is actually the most interesting piece of the whole project, if you care about building things on the internet rather than just using them. Every domain anyone scans doesn't just answer that one person's question — it becomes a permanent, indexable, SEO-friendly page. Scan example.com once, and there's now a page at sitegraph.tech/site/example-com that anyone, including Google, can find forever.

That means the content engine grows entirely on its own, driven purely by usage. Nobody has to sit down and write ten thousand articles about ten thousand websites — the act of someone being curious about a domain is the content creation event. It's programmatic SEO in its most honest form: every page exists because someone genuinely wanted the information on it, not because it was mass-generated to bait a search query.

Why It's Actually Free

There's no asterisk here. SiteGraph runs on zero paid third-party APIs or data dependencies, anywhere in the stack. Every signal — DNS, TLS, HTTP, HTML, WHOIS/ASN — comes from data that's already public and free to fetch directly. That's a deliberate constraint, not an accident: it's what lets the whole thing stay free to use at any scale, and free for anyone to self-host from the open repository if they'd rather run their own copy than trust ours.

Who This Is Actually For

• Developers comparing frameworks or checking what a site they admire is built with
• Founders and marketers scoping a competitor's stack, hosting, and SEO setup before a pitch or a rebuild
• Security-minded teams spot-checking whether HSTS, CSP, and cookie flags are actually configured, not just assumed
• SEO practitioners auditing meta tags, structured data, and robots/sitemap health on their own or a client's site
• Anyone curious — genuinely, this is the use case we didn't plan for and see the most. People just like knowing how things are built.

What's Next

The signature library keeps growing — more frameworks, more CMS platforms, more e-commerce and analytics tools recognized with every release. Security scoring is getting more granular, and the comparison and country/hosting graph views are getting deeper as the index of scanned sites keeps growing past the six-thousand mark. None of it changes the core promise: point it at a domain, get the real answer, for free.

Try It Yourself

If you've ever wondered what a website is actually running on, what's protecting it, or who's hosting it — stop guessing from View Source. Go to sitegraph.tech, type in any domain, and see the whole thing laid bare in a few seconds. Then go find a site you didn't expect to see on the same CDN as you, and follow the graph from there.