How to Build a Subdomain Inventory From URLs

Turn crawl exports, logs and configuration lists into a hostname-level subdomain inventory for migrations, security and content mapping.

FlashDomains Editorial·August 21, 2026·8 min read

A subdomain inventory answers a different question from a root-domain list. Instead of reducing every service to example.com, it preserves hostnames such as shop.example.com, api.example.com and status.example.com. That distinction matters during migrations, incident response and platform consolidation.

What belongs in a useful inventory?

Field	Example	Why retain it
Hostname	`api.eu.example.com`	Primary inventory key
Root domain	`example.com`	Groups related hosts
Subdomain labels	`api.eu`	Shows hierarchy
Source and line	`crawl.csv:42`	Provides evidence
Occurrence count	`18`	Helps prioritize review

How to build the first-pass inventory

Collect source URLs. Combine crawler exports, reverse-proxy logs, sitemap files, analytics hostnames and maintained configuration lists.
Select hostname mode. Root mode intentionally removes the distinction you are trying to map.
Do not strip meaningful labels. Stripping conventional www may be acceptable; keep all other subdomains.
Deduplicate with counts. One hostname becomes one row while frequency records its visibility in the supplied sources.
Export and enrich separately. Add DNS, owner, environment, certificate, response and migration status fields in your inventory system.

Repeatable process

Build a maintainable hostname inventory

Preserve subdomains, attach evidence and separate discovery from ownership decisions.

1
Combine approved source lists
Use crawl exports, logs, DNS records, certificates and team-maintained inventories where authorized.
2
Extract full hostnames
Keep every label before the registrable domain instead of collapsing to root-domain output.
3
Normalize and deduplicate
Standardize case and internationalized names, then group identical hostnames with source counts.
4
Classify and verify
Add owner, purpose, environment, response status and last-observed fields outside the extractor.

Side-by-side

Discovery sources reveal different parts of the estate

No single source should be treated as a complete or continuously current inventory.

Decision	Strength	Blind spot
Crawler	Linked public web hosts	Unlinked, private and authenticated services
Traffic logs	Hosts used during the retention window	Inactive or never-requested hosts
DNS records	Configured names in controlled zones	External services and stale records
Certificate data	Names present in observed certificates	Non-TLS services and disclosure limitations

Where it helps

What a hostname inventory supports

Use the cleaned output as a focused input to the next analysis—not as a substitute for that analysis.

Site migrations

Identify shops, help centers, APIs and regional sites that need separate migration plans.

Certificate planning

Review which active hosts require TLS coverage and ownership confirmation.

Attack-surface review

Create a permitted triage list for exposed hosts and forgotten environments.

Platform ownership

Assign operational owners and lifecycle status to discovered services.

Worked example

Keep operational boundaries visible

Input

https://www.example.com
https://api.eu.example.com/v2
https://shop.example.com/products
https://status.example.com

Result

www.example.com
api.eu.example.com
shop.example.com
status.example.com

Root-domain mode would collapse every row to example.com and hide the service boundaries.

No single export proves completeness. A public crawler cannot see hosts that require authentication or are not linked. DNS data can include inactive records. Logs only reveal traffic during the retained period. Certificate records and passive datasets have their own scope and policy constraints.

Record where each hostname came from and when it was observed. The extractor’s multi-source mapping helps establish that first layer, but it does not query DNS or certify that a host is live.

Migration use case

Assign every hostname an owner, destination, redirect plan, certificate requirement, DNS change and validation status. Separating www, store, app, API, support and regional hosts prevents a root-domain redirect plan from overlooking operational services.

Security use case

Compare observed hosts with the approved inventory. Unknown does not automatically mean malicious—it may be a forgotten vendor, preview environment or stale record—but it deserves ownership and exposure review. Preserve exact hostnames and original log context.

Content and SEO use case

A hostname inventory can reveal content split across blogs, stores, help centers and international properties. It is a starting map, not an indexation report. Response codes, canonicals, robots directives and Search Console data must be evaluated separately.

Root, hostname and nested subdomain

For api.eu.example.co.uk, the root is example.co.uk and the subdomain portion is api.eu. Read root domain versus hostname versus subdomain and the parts of a domain before designing inventory columns.

Frequently asked questions

Does this enumerate subdomains from DNS?

No. It extracts hostnames present in content you supply. DNS enumeration is a separate, permission-sensitive discovery task.

Can it preserve nested subdomains?

Yes. Hostname mode keeps every label in values such as api.eu.example.com.

Should `www` remain in the inventory?

Keep it when it resolves, redirects or is managed separately. Remove it only when your inventory intentionally treats it as equivalent to the apex host.